Why Make Ethereum Blockchain Data Available on Google Cloud? 6715

Ethereum and other cryptocurrencies have captured the imagination of technologists, financiers, and economists. Digital currencies are only one application of the underlying blockchain technology. Earlier this year, we made the Bitcoin dataset publicly available for analysis in Google BigQuery. Today we’re making the Ethereum dataset available.

Like its predecessor, Bitcoin, you might think of the Ethereum blockchain as an immutable distributed ledger. However, creator Vitalik Buterin extended its set of capabilities by including a virtual machine that can execute arbitrary code stored on the blockchain as smart contracts.

With regards to the system architecture, Ethereum resembles Bitcoin in that it primarily serves to record immutable transactions. Both are essentially OLTP databases, and provide little in the way of OLAP (analytics) functionality. However, the Ethereum dataset is notably distinct from the Bitcoin dataset:

  • The Ethereum blockchain’s primary crypto-economic unit of value is Ether, while the Bitcoin blockchain’s unit of value is Bitcoin. However, the majority of value transfer on the Ethereum blockchain is composed of so-called tokens. Tokens are created and managed by smart contracts.
  • Ether value transfers are precise and direct, resembling accounting ledger debits and credits. This is in contrast to the Bitcoin value transfer mechanism, for which it can be difficult to determine the balance of a given wallet address.
  • Addresses can be not only be wallets that hold balances, but can also contain smart contract bytecode that allows the programmatic creation of agreements and automatic triggering of their execution. An aggregate of coordinated smart contracts could be used to build a decentralized autonomous organization.

The Ethereum blockchain data are now available for exploration with BigQuery. All historical data are in the ethereum_blockchain dataset, which updates daily. The Ethereum ETL project on GitHub contains all source code used to extract data from the Ethereum blockchain and load it into BigQuery—we welcome more contributors and more blockchains!

Why make Ethereum blockchain data available on Google Cloud?

While the Ethereum blockchain peer-to-peer software has an API for a subset of commonly used random-access functions (for example: checking transaction status, looking up wallet-transaction associations, and checking wallet balances, for example), API endpoints don’t exist for easy access to all of the data stored on-chain.

Perhaps more importantly, API endpoints also don’t exist for viewing the blockchain data in aggregate.  Here’s an example chart that shows the total Ether transferred and average transaction cost, aggregated by day:

Ether aggregate transfers and transaction costs

A visualization like this (and the underpinning database query) is useful for making business decisions, such as prioritizing improvements to the Ethereum architecture itself (is the system running close to capacity and due for an upgrade?) to balance sheet adjustments (how quickly can a wallet be rebalanced?).

BigQuery has strong OLAP capabilities to support this type of analysis, ad-hoc and in general, without requiring additional API implementation.

Accordingly, we built a software system on Google Cloud that:

  1. Synchronizes the Ethereum blockchain to computers running Parity in Google Cloud.
  2. Performs a daily extraction of data from the Ethereum blockchain ledger, including the results of smart contract transactions, such as token transfers.
  3. De-normalizes and stores date-partitioned data to BigQuery for easy and cost-effective exploration.

The Ethereum blockchain dataset is also available on Kaggle here. You can query the live data in Kernels, Kaggle’s no charge in-browser coding environment, using the BigQuery Python client library. Fork this example kernel to experiment with your own copy of the Python code.

Interesting queries and analyses

Below, we show a number of interesting queries and visualizations based on the Ethereum dataset. Our analyses focus on three popular topics:

  • Smart contract function calls
  • On-chain transaction time-series and transaction networks
  • Smart contract function analytics

Analysis 1: Popular Smart Contracts Event Logs

The main use case for the Ethereum blockchain has so far been the exchange of digital tokens. Below we demonstrate querying the dataset’s transactions and contracts tables to find the most popular smart contracts, as measured by transaction count:

What are the 10 most popular Ethereum collectibles (ERC-721 contracts), by number of transactions? View my query here.

Contract addresses and transfer counts for the 10 most popular Ethereum collectibles 

The most popular ERC-721 smart contract by transaction count is 0x06012c8cf97bead5deae237070f9587f8e7a266d, the main smart contract for the CryptoKitties game. We’ll take a look at some attributes of the game later in this document.

Looking more closely at this contract’s source code, it logs a CryptoKitty birth event to the blockchain. You can query the logs table for instances of this event here.

We can visualize the CryptoKitty pedigree, as shown here for accounts that own at least 10 CryptoKitties. Color indicates owner, while size indicates the PageRank (reproductive fitness) of each CryptoKitty:

CryptoKitty PageRank based on game's contract's source code

Analysis 2: Transaction Volumes and Transaction Networks

There are many types of tokens being distributed on the Ethereum blockchain, and their patterns of distribution vary by type and also by time. By looking at each token’s transaction activity, we can measure which are more popular in aggregate or within a given time frame.

Here’s a query to measure an aggregate token statistic: which are the 10 most popular Ethereum tokens (ERC20 contracts), by number of transactions? Find the answer here.

At position #5, one of the most popular tokens is OmiseGO ($OMG) at address 0xd26114cd6ee289accf82350c8d8487fedb8a0c07.

As a follow-up, here’s a query for measuring a token statistic (transaction count) by time window, specifically the daily count of $OMG tokens transfers, and a Data Studio visualization of these data as a time series from inception through August 2, 2018 from this Google Sheet:

OmiseGO exchange daily transfers

Note that on September 13, 2017, there was a large increase in the number of $OMG receivers but no increase in the number of senders. This corresponds to the beginning of the OmiseGO Token Airdrop.

Because the data, at a highly granular level, consists of a set of transfers between wallet addresses, we can also reason about the data using a directed graph data structure.

Here’s a visualization of a subset of the same data: it’s the first 50,000 transactions that had at least two trading partners. In this graphic, nodes (points) represent wallet addresses on the Ethereum blockchain, and edges (lines) represent aggregate transfer of tokens between a pair of addresses. Edge length is approximately proportional to the number of tokens transferred, meaning wallets that transfer more aggregate tokens between them are closer together in the chart. Further groups of addresses that transfer often with one another—to the exclusion of members of other groups—will cluster together, and we’ve color coded these groups for clarity. This graphic was made with Gephi and nodes are color-labeled by groups calculated using the Modularity algorithm.

Transfer graph visualization of Ethereum transactions

Analysis 3: Analysis of Smart Contract Functionality

We mentioned previously that many of the smart contracts on the Ethereum blockchain are ERC-20 contracts. What does this mean? ERC-20 simply defines a software interface that smart contracts can implement. Specifically, it’s composed of a few functions related to token transfer, fully described in the ERC20 Token Standard specification document.

There are many other functions that can be implemented by a smart contract. Fortunately, many smart contracts’ source code is freely available to use (open source). We can use this to gain some knowledge about what other contracts do from the name of the function, even for those of which we don’t have the source code, because common function names will share a common signature.

Coming back to the CryptoKitties, discussed in Analysis 3 above, the major element of gameplay is animal husbandry, and the mixing of genes in a breeding event is implemented in the CryptoKitties GeneScience smart contract, 0xf97e0a5b616dffc913e72455fde9ea8bbe946a2b. Suppose we wanted to find other games also implementing similar gameplay mechanics to the CryptoKitties GeneScience contract? We can measure this using a JavaScript UDF implementation of the Jaccard similarity coefficient in this query.

These results reveal that several earlier versions of the GeneScience contract are most similar to the current version of the smart contract at address 0xf97e0a5b616dffc913e72455fde9ea8bbe946a2b. But there are also some others (e.g. CryptoPuppies at 0xb64e6bef349a0d3e8571ac80b5ec522b417faeb6), that appear to be highly similar contracts, as measured by method signatures.

Previous ArticleNext Article

Symmetry Launches Revolutionary Platform for Decentralized Crypto Indices and Actively Managed Funds on Solana 2250

Today marks a groundbreaking step for decentralized finance (DeFi) on the Solana blockchain as Symmetry launches its highly anticipated User Interface at https://app.symmetry.fi/.

The platform aims to revolutionize the way users manage their portfolios through crypto indices and actively managed funds by providing an all-in-one solution for creating, managing, buying, and selling funds.

Symmetry is a decentralized Crypto Indices and Actively Managed Funds infrastructure layer built on Solana, powered by the Symmetry Engine. This innovative on-chain asset management infrastructure covers everything from on-chain funds, indices, and multi-token liquidity pools, to liquidity routing between indices and decentralized exchange (DEX) aggregators, and public APIs for other DeFi projects to integrate Symmetry products seamlessly.

The platform offers a wide range of opportunities for both fund managers and users. Managers can create and manage funds with multiple tokens that reweigh, rebalance, and refilter according to custom rules defined by the manager, or create a trustless Crypto Index with predefined rules. Users can create their own actively managed funds or indices and buy and sell funds created by other users or protocols.

Symmetry’s liquidity provision feature allows indices and actively managed funds to act as liquidity providers on DeFi aggregators such as PRISM and Jupiter when their token weights deviate from the target weights. This groundbreaking feature not only enables funds to rebalance at zero cost but also generates fees from aggregator users for fund managers and holders, a significant departure from traditional approaches where indices and funds typically pay fees on exchanges to rebalance.

The Symmetry Engine relies on Pyth, a reliable price oracle, to determine true prices for each asset utilized in Symmetry products. This is crucial for funds to accurately determine fund values, rebalancing triggers, and buy/sell values for users.

Example use-cases for Symmetry include decentralized index fund apps, decentralized fund management apps, treasury management tools, retail onboarding apps, copy-trading apps, and investing apps. The platform envisions a future where anyone can create an index fund, manage on-chain funds and portfolio strategies, convert wallet portfolios to funds, and integrate index or actively managed fund tokens for trading.

With the launch of the Symmetry User Interface, the world of decentralized finance on the Solana blockchain is poised to experience a paradigm shift in how users interact with and manage their crypto portfolios. As the platform continues to roll out its full suite of features to all users, the future of Solana DeFi looks brighter than ever.

New DeFi Protocol, FORGE․SX, Offers Stocks and Other Synthetic Assets on Blockchain Ecosystem 2519

Individuals seeking to trade real world assets on the blockchain will have a new ally with the launch of the Forge.SX DeFi protocol.

With the convenience of a blockchain-based platform, Forge.SX empowers anyone to mint and trade synthetic versions of real-world assets with the convenience of blockchain technology. There are a number of assets available in the protocol, including commodities, stocks, and even foreign currencies. Popular stocks such as fTSLA are already available on the platform. Several plans are already in motion for the addition of additional stocks in the future.

According to the core team, the reason for offering mint trade of stocks and other synthetic assets is as follows: “As seasoned crypto investors, we understand the value and potential of synthetic assets on the blockchain and want everyone to have access to global financial markets and mint synthetic digital assets regardless of their financial status. Synthetic investments like stocks are among the most valuable. Although risks are involved in stock investment, because the market is regulated, they offer greater reassurance for cryptocurrency investors. We also wanted to diversify the assets available on the platform to attract new investors who are interested in yielding returns with Real World Assets.”

The assets on Forge are called fAssets and can be forged and burned in the Ovens by anyone. As standard ERC-Tokens, assets can only be minted against collateral deposits. Traders can begin by opening an oven using USDC as collateral for the creation of synthetic assets. The crypto trader may then burn the fAssets minted by closing the oven, and they will be able to recover the collateral that has been locked.

Forge.SX is supported by a reliable Oracle data feed that connects the world of Blockchain to the Forge.SX platform. A competent and experienced team is also available at the company, with a strong background in Dapp development and DeFi. In order to provide traders with the best service in the industry, Forge.SX consistently performs research and development. Core Forgers said: “We do our finest to bring safety and value to our investors and remain transparent to inspire confidence in our business model and the future of synthetic assets.”

To learn more about Forge or to start trading real-world assets such as stocks on Forge SX, please visit: https://forge.sx/

$C4 Investing, a unique business model 2316


The Cardano Crocs Club is an innovative project that has created a token-based economy through staking on the Cardano blockchain. However, as more projects start staking and creating tokens, the purpose of staking can become unclear. To turn a token into a widely-used currency, it is important to create a real-world use case for the token that goes beyond simply staking it for rewards.

Net Technology, the parent company of the Cardano Crocs Club, has taken a unique approach by investing real EURO into ready-made projects, which are then developed. This creates a real business ecosystem that uses the C4 token as a means of exchange and value preservation. By using the token as a payment method for all ready-made projects, there is a real use case for the token that can potentially drive adoption and demand for it.

Creating a network effect is crucial for driving the adoption of a token. When more businesses and projects accept a token as a payment method, it can increase the token’s visibility and perceived value. This, in turn, can lead to more people wanting to hold and use the token, which can potentially increase its value over time.

Net Technology is also committed to creating products and services that will continue to provide value in the future. This can help to ensure the longevity of the ecosystem and the token. By backing sustainable and profitable businesses, they are creating a solid foundation that can support the growth and adoption of the C4 token.

In conclusion, the Cardano Crocs Club is taking a unique and promising approach to creating a token-based economy that goes beyond simple staking. By investing in ready-made projects, creating a real business ecosystem, and backing sustainable and profitable businesses, they are creating a real-world use case for the C4 token. By driving adoption and demand for the token, they are creating the potential for it to become a widely-used currency.

OpenOcean Pioneers Efficient Crypto Swaps on 23 Chains, integrated by MetaMask 2473

OpenOcean is excited to announce that MetaMask, the most popular non-custodial DeFi wallet, has integrated OpenOcean’s advanced swaps for their in-wallet trades. Swaps will be supported on six major public blockchains – Ethereum, BNB Chain, Polygon, Arbitrum, Optimism and Avalanche.

OpenOcean is the leading DEX Aggregator in web3 and crypto. OpenOcean indexes vast liquidity for trades and automatically routes users the best possible trade with the lowest slippage and best final swap returns. OpenOcean is backed by top investors – Binance Labs, CMS, Kenetic, Altonomy, Multicoin and more. OpenOcean is proud to support over 23+ blockchains and over 280+ Decentralized exchanges and liquidity pools.

As a primary entry point for many crypto users, MetaMask users can now access OpenOcean’s superior and efficient trades across all supported chains. The integration with OpenOcean has enabled traders to access a vast ocean of liquidity at competitive prices. This collaboration represents a significant milestone for both parties. OpenOcean can now benefit more DeFi users with its unique trade routing algorithm by split-routing trades across different decentralized exchanges for the best possible efficiency.

“This integration is yet another step in our mission to bring the best service and prices to MetaMask Swaps users.” From MetaMask Wallet team. (Source – https://twitter.com/MetaMask/status/1650856215977549824?s=20)

As a middleware protocol, OpenOcean has published its free-to-use API and SDK which is supported on 19 EVM chains. OpenOcean offers the best environment for secure, stable, and competitive swap rates for all developers and users. In addition to MetaMask, OpenOcean’s API has been adopted by leading protocols such as DefiLlama, Ankr, Fantom Wallet, Li Finance, Gnosis safe, Vesta Finance, Sonne Finance, etc.

About OpenOcean

OpenOcean is a pioneering middleware developer, dex aggregator, cross-chain swap aggregator and Web 3 developer in the crypto space, offering a suite of tools across 23+ networks and 280+ deep liquidity sources. OpenOcean is building a one-stop trading platform, to bring users the best swap return for decentralized crypto trading.

Website: https://openocean.finance/
Twitter: https://twitter.com/OpenOceanGlobal
Discord: https://discord.gg/C7PHQaKdNX
Telegram: https://t.me/OOFinance

GMX Votes to Integrate Chainlink’s New Low-Latency Oracles as Launch Partner, in Major Leap Forward for DeFi Innovation and Economic Sustainability 2163

GMX, the largest decentralized perpetual exchange operating on Arbitrum and Avalanche, and Chainlink, the industry-standard Web3 services platform, today announced that the GMX community has voted to integrate Chainlink’s new low-latency oracles for financial market data. GMX will join as the launch partner of the new solution, integrating the low-latency oracles in its upcoming protocol GMX V2 to support perpetuals, swaps and liquidity.

The low-latency oracles are a new standard in ultra-fast oracle infrastructure, and serve as a major step forward for improving the security, decentralization, and user experience of the DeFi economy. The proposal can be found on the GMX snapshot page.

Chainlink’s new low-latency oracles are a major innovation that address a significant DeFi pain point — the extreme latency-sensitivity of derivatives exchanges and other price-sensitive DeFi applications. Low-latency oracles address this concern by providing secure access to high-quality market data in a pull-based manner, all while keeping Chainlink’s industry standard and battle-tested infrastructure, where data is cryptographically signed by a decentralized oracle network and verified on-chain.

In the case of GMX, the integration is set to enhance the decentralized exchange’s performance, harden its data security, help mitigate the risks of frontrunning, and provide long-term sustainable access to essential Web3 infrastructure. The Arbitrum testnet now has a beta version of the low-latency oracle deployed, and testing by GMX core contributors is ongoing.

“We’re excited to see our community approve this integration and for GMX to be the launch partner of Chainlink’s new low-latency oracles,” said X, development core contributor of GMX. “GMX contributors have been working with Chainlink Labs since last year on the specifications of these new oracles, and we’re excited to contribute to establishing a long-term self-sustainable oracle network.”

The collaboration between GMX and Chainlink also marks a transformational moment for the maturity and economic sustainability of Web3 through the creation of a sustainable financial framework. A fee structure will be established in which 1.2% of total fees generated by GMX V2 and later protocol versions will be paid to Chainlink service providers in exchange for the usage of the low-latency oracles and technical support.

“Chainlink Labs has been committed to the support and growth of DeFi since its very early days,” said Johann Eid, VP of Go-To-Market at Chainlink Labs. “With this collaboration, we mark a new milestone for the space and its race towards mass adoption. The low-latency oracles will bring the industry one step closer to the performance level that currently exists outside of it, while our economic alignment helps set the foundation for a more sustainable ecosystem. We’re excited to keep building this space with leading projects such as GMX, one block at a time until DeFi becomes a leading and secure global financial system.”

The passing of this proposal represents a major step forward for oracle sustainability as Chainlink continues to deliver on its Economics 2.0 vision, following other programs such as SCALE and BUILD.

As the Chainlink Staking protocol evolves over time to support a greater scope of data feeds and oracle services, a portion of the fees from this proposal are planned to be sent directly to stakers as user fee rewards for their work increasing the crypto-economic security of Chainlink services utilized by GMX.

About Chainlink

Chainlink is the industry-standard Web3 services platform that has enabled trillions of dollars in transaction volume across DeFi, insurance, gaming, NFTs, and other major industries. As the leading decentralized oracle network, Chainlink enables developers to build feature-rich Web3 applications with seamless access to real-world data and off-chain computation across any blockchain, and provides global enterprises with a universal gateway to all blockchains.

Learn more about Chainlink by visiting chain.link or reading the developer documentation at docs.chain.link.

About GMX

GMX is a permissionless decentralized spot and perpetual exchange. Traders benefit from low fees, deep liquidity, reduced liquidation risks and transparent pricing. Liquidity on GMX is user-contributed, and liquidity providers earn a majority of the fees the protocol generates, without external market makers.

GMX has worked collaboratively to create a robust ecosystem, with over 100 DeFi integrations. With GMX V2, any asset with a Chainlink low-latency price feed may be transacted on the exchange, and order execution is guaranteed, further optimizing the on-chain trading experience. Discover GMX at gmx.io.

Lambda256 Unveils Luniverse NOVA, the Blockchain Node Service for Web3 Developers 3229

Lambda256, the blockchain tech arm of South Korean fintech leader Dunamu, has launched a new web3 development service platform named Luniverse NOVA. The platform aims to provide a user-friendly way for developers to build decentralized applications (dApps) on public chains such as Ethereum and Polygon networks, with advanced node service and high interoperable APIs.

Luniverse NOVA comes equipped with a range of features to enhance the efficiency and security of blockchain development and operation. The service offers real-time blockchain node services, multichain API, statistics API, webhook API, and Stream API, which supports different kinds of nodes, including Ethereum and Polygon. The service also provides automatic load balancing, ensuring that nodes remain online and functional during sudden spikes in traffic.

In addition to its advanced features, Luniverse NOVA offers competitive pricing options, with pricing based on factors such as the number of nodes and the amount of data request calls. Celebrating the groundbreaking launch of Luniverse NOVA, Lambda256 has introduced a special promotion for its users. For a limited time, exclusive price discounts on the subscription fees and a limited USDC prize and membership NFT will be given to new users who register for the platform as early birds.

As the only partner in Korea of Polygon DevX APAC Roadshow, Luniverse will participate in upcoming Polygon Global Hackathon as well. “We’re excited to launch Luniverse NOVA and offer not only simple but also reliable way to build dApps to Polygon community,” said a spokesperson for Lambda256. “Developers can build decentralized applications (dApps) on Polygon networks without having to worry about the complexities of node settings and management. With its user-friendly interface, advanced features, and competitive pricing, we believe Luniverse NOVA will become a popular choice for developers looking to take advantage of the Polygon ecosystem.”

With the launch of Luniverse NOVA, Lambda256 aims to further strengthen its position as a leading provider of blockchain technology solutions. The company has been actively working on developing new products and services to help drive the adoption of blockchain technology across various industries. More details of the Luniverse NOVA can be found on the official website.

About Luniverse

Luniverse is a Web3 infrastructure service platform that offers innovative technology such as a development environment, API, and operation tools that allow users to quickly and easily create blockchain services. The development environment allows all developers and companies to apply blockchain technology to businesses and enter the Web3 space without difficulty. Highly efficient blockchain services can be developed and operated with ease thanks to simple network settings, convenient operating tools, and high-performance chains of over 3000TPS. Aiming to become an eco-friendly blockchain, Luniverse dramatically reduced power consumption that can occur during virtual asset mining. As a result, it received certification from DNV, a third-party international organization, and also joined the CCA in working toward a goal of low carbon emissions by 2040.

In 2023 Luniverse will upgrade to Luniverse NOVA, a new version of Luniverse that includes all services required to build Web3, NFT, and other blockchain services. It will introduce blockchain node services, Multichain API for Ethereum and Polygon blockchain development, as well as a dashboard and data indexing features required for easy blockchain service operation.