[2022/11/28] Updates on Ethereum data

Hey team, ZettaBlock is launching new changes to improve the decoding rate from 80% to 96% for Ethereum logs

TL;DR

Schema updates

By 2022/11/30, the raw and decoded Ethereum tables will be merged. The tables affected are:
- ethereum_mainnet.logs
- ethereum_mainnet.traces
- ethereum_mainnet.transactions
By 2022/12/02, the decoded Ethereum tables will no longer be supported.
- ethereum_mainnet.decoded_logs
- ethereum_mainnet.decoded_traces
- ethereum_mainnet.decoded_transactions

Improvement effects

The decoding rate of raw Ethereum logs will be improved from 80% to 96%
Support decoding logs, transactions, and traces without contract ABI

Details

Dear ZettaBlock users,

Based on the feedback we have received , we are launching a set of changes to our Ethereum data both in structure and data richness. We believe that these changes will keep our table structure comprehensive and straightforward, as well as make the process of querying our data easier. Please make adjustments to your queries, dashboards, and user APIs accordingly to avoid interruption.

In the effort to reduce data redundancy and usage friction, we are merging the tables for raw and decoded Ethereum data under database ethereum_mainnet. After the changes, data previously scattered in 6 tables will be hosted by the 3 tables as shown below.

Before the change	After the change
`logs`, `decoded_logs`	`logs`
`traces`, `decoded_traces`	`traces`
`transactions`, `decoded_transactions`	`transactions`

By 2022/11/30, we will horizontally expand table logs, traces, and transactions to include columns from the respective decoded tables. For detailed schema changes, please check out the Appendix.

By 2022/12/02, we will sunset the old decoded tables.

ethereum_mainnet.decoded_logs
ethereum_mainnet.decoded_traces
ethereum_mainnet.decoded_transactions

With a new decoding system, we will bring a significant improvement to the decoding rate of Ethereum data. Based on the internal calculation, the decoding rate of raw logs will be improved to 96% compared to the previous 80%.

Previously, only data for smart contracts either verified on EtherScan or having complete ABI was decoded. After the schema changes, rows decoded by complete contract ABI will be indicated by having column decoded_from_abi as true.
With the new decoding system, we decode logs, transactions, and traces based on the Keccak hash (SHA3) of the function or event if the complete contract ABI is unavailable. For rows decoded by this approach, the value of column decoded_from_abi is false.

Appendix

ethereum_mainnet.logs

Before the change	After the change
transaction_hash: string	transaction_hash: string
transaction_index: bigint	transaction_index: bigint
block_number: bigint	block_number: bigint
block_hash: string	block_hash: string
removed: boolean	removed: boolean
log_index: bigint	log_index: bigint
data: string	data: string
topics: array	topics: array
contract_address: string	contract_address: string
anonymous: boolean	anonymous: boolean
	event: string
	event_signature: string
	argument_names: array
	argument_types: array
	argument_values: array
block_time: timestamp	block_time: timestamp
	text_signature: string
	decoded_from_abi: boolean
process_time: timestamp	process_time: timestamp
data_creation_date: date	data_creation_date: date

ethereum_mainnet.traces

Before the change	After the change
transaction_hash: string	transaction_hash: string
transaction_index: bigint	transaction_index: bigint
type: string	type: string
block_number: bigint	block_number: bigint
block_hash: string	block_hash: string
block_time: timestamp	block_time: timestamp
from_address: string	from_address: string
to_address: string	to_address: string
value: decimal(27, 0)	value: decimal(27, 0)
input: string	input: string
output: string	output: string
trace_type: string	trace_type: string
call_type: string	call_type: string
reward_type: string	reward_type: string
gas: decimal(21, 0)	gas: decimal(21, 0)
gas_used: bigint	gas_used: bigint
subtraces: bigint	subtraces: bigint
trace_address: array	trace_address: array
error: string	error: string
status: bigint	status: bigint
transaction_status: bigint	transaction_status: bigint
	func_name: string
	func_signature: string
	argument_names: array
	argument_types: array
	argument_values: array
	output_parameters: string
	output_names: array
	output_types: array
	output_values: array
trace_id: string	trace_id: string
trace_index: bigint	trace_index: bigint
	text_signature: string
	decoded_from_abi: boolean
process_time: timestamp	process_time: timestamp
data_creation_date: date	data_creation_date: date

ethereum_mainnet.transactions

Before the change	After the change
hash: string	hash: string
nonce: bigint	nonce: bigint
block_hash: string	block_hash: string
block_number: bigint	block_number: bigint
transaction_index: bigint	transaction_index: bigint
from_address: string	from_address: string
to_address: string	to_address: string
value: decimal(27, 0)	value: decimal(27, 0)
type: string	type: string
gas_price: decimal(21, 0)	gas_price: decimal(21, 0)
input: string	input: string
v: string	v: string
s: string	s: string
r: string	r: string
max_fee_per_gas: decimal(21, 0)	max_fee_per_gas: decimal(21, 0)
max_priority_fee_per_gas: decimal(21, 0)	max_priority_fee_per_gas: decimal(21, 0)
chain_id: string	chain_id: string
access_list: array	access_list: array
gas_limit: bigint	gas_limit: bigint
	func_name: string
	func_signature: string
	argument_names: array
	argument_types: array
	argument_values: array
block_time: timestamp	block_time: timestamp
status: bigint	status: bigint
gas_used: bigint	gas_used: bigint
cumulative_gas_used: bigint	cumulative_gas_used: bigint
effective_gas_price: decimal(21, 0)	effective_gas_price: decimal(21, 0)
	text_signature: string
	decoded_from_abi: boolean
process_time: timestamp	process_time: timestamp
data_creation_date: date	data_creation_date: date