Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upstream: merge 2.61.2 to fork #1804

Draft
wants to merge 262 commits into
base: zkevm
Choose a base branch
from

Conversation

IvanBelyakoff
Copy link
Collaborator

WIP: merge Erigon 2.61.2 to zkevm

manav2401 and others added 30 commits July 23, 2024 18:05
…erigontech#11295)

Cherry-picked from erigontech#11294

> Remove the checks for amoy added previously to prepare for mainnet
release.
> Sets `txpool.pricelimit`, `miner.gasprice` and `gpo.ignoreprice` to
25gwei for all polygon based networks.
Merging diagnostics code updates from main branch.
Removed unnecessary struct type

Co-authored-by: Somnath <[email protected]>
- Updated gopsutil version as it has improvements in getting processes
and memory info.
…ch#11549) (erigontech#11551)

cherry-pick 2a98f6a for E2

relates to:
erigontech#10734
erigontech#11387

restart Erigon with `SAVE_HEAP_PROFILE = true` env variable wait until
we reach 45% or more alloc in stage_headers when "noProgressCounter >=
5" or "Rejected header marked as bad"
Changes:
- Collecting CPU and Memory usage info about all processes running on
the machine
- Running loop 5 times with 2 seconds delay and to calculate average
- Sort by CPU usage
- Write result to report file
- Display totals for CPU and Memory usage to processes table
- Display CPU usage by cores
- Print CPU details to table

Cherry pick from:
- erigontech#11516
- erigontech#11526
- erigontech#11537
- erigontech#11544
- added flags which was applied to run command to report
Fixed issue with setup diagnostics client on erigon start
…erigontech#11669)

I have done specific integration test(on e2) and compare the result with
etherscan (block number 0x3D08FF: 3999999).
etherscan link:

https://etherscan.io/tx/0x535bd65cfee8655a1ffd8a28e065b47bcf557b10641491a541ab1dd08496709e#statechange
for account 0xEce701C76bD00D1C3f96410a0C69eA8Dfcf5f34E if use
block_number + 1 the debug_accountRange() returns correctly for storage
0x0000000000000000000000000000000000000000000000000000000000000004 the
new value:
0x000000000000000000000000000000000000000000000064a66fad67042ceb4a;
using current software returns the old value
(0x000000000000000000000000000000000000000000000064a3a922764918eb4b).
debug_accountRange() returns :
 "storage": {
     ......
"0x0000000000000000000000000000000000000000000000000000000000000004":
"64a66fad67042ceb4a",
      ----
     }
 If approved I will make change for e3
…ontech#11649)

- Added functionality to grab the heap profile by calling the
diagnostics endpoint
- Added support to pass heap profile file to diagnostics UI through
WebSocket
shohamc1 and others added 23 commits February 1, 2025 23:36
This PR introduces a new GitHub Actions workflow,
`qa-sync-from-scratch-externalcl.yml`, designed to validate Erigon's
ability to sync from scratch and maintain synchronization using an
external consensus layer (Prysm). The workflow runs on `release/2.*`
branches and supports manual execution via `workflow_dispatch`.
Previously due to a change the prefix was getting omitted from built
payload
Rhe tip-tracking test has been replaced by the sync-with-externalcl test
…tech#13774)

This PR updates the **QA sync workflow** to improve flexibility and
expand compatibility with different consensus clients and networks.

#### Key Changes:
1. **Renamed Job**:  
- Changed `prysm-minimal-node-sync-from-scratch-test` →
`sync-with-externalcl` for broader applicability.

2. **Matrix Strategy for Multiple Clients & Chains**:  
   - Introduced `matrix` configuration to run tests for:
     - Clients: **Lighthouse, Prysm**
     - Chains: **Mainnet, Gnosis**
   - **Exclusion**: Prysm + Gnosis is **not** tested.

3. **Client-Specific Setup**:  
- Lighthouse and Prysm are now dynamically installed based on the matrix
configuration.
   - JWT secret is generated per test run.

4. **Erigon Sync Execution Update**:  
- Uses `${{ matrix.chain }}` and `${{ matrix.client }}` dynamically in
test execution and result logging.

5. **Enhanced Logging & Artifacts**:  
   - Client-specific logs are now uploaded as artifacts.
   - Improved log handling for both Lighthouse and Prysm.

6. **Additional Cleanup Steps**:  
- Removed consensus layer data (`CL_DATA_DIR`) after tests to free up
disk space.
Diable test that are failing due to:
- Pectra changes (eth_feeHistory)
- totalDifficulty removal (eth_getBlockByHash, eth_getBlockByNumber)
Copy link

cla-bot bot commented Mar 7, 2025

We require contributors/corporates @manav2401, @VBulikov, @lupin012 to read our Contributor License Agreement, please check the Individual CLA document/Corporate CLA document


func newContext(prefix []interface{}, suffix []interface{}) []interface{} {
normalizedSuffix := normalize(suffix)
newCtx := make([]interface{}, len(prefix)+len(normalizedSuffix))

Check failure

Code scanning / CodeQL

Size computation for allocation may overflow High

This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.

Copilot Autofix AI 3 days ago

To fix the problem, we need to ensure that the combined length of prefix and normalizedSuffix does not exceed the maximum value for the type used in the allocation. We can achieve this by adding a guard condition before performing the allocation. If the combined length exceeds a safe limit, we should handle this case appropriately, such as by returning an error.

Suggested changeset 1
erigon-lib/log/v3/logger.go

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/erigon-lib/log/v3/logger.go b/erigon-lib/log/v3/logger.go
--- a/erigon-lib/log/v3/logger.go
+++ b/erigon-lib/log/v3/logger.go
@@ -140,2 +140,5 @@
 	normalizedSuffix := normalize(suffix)
+	if len(prefix) > (int(^uint(0) >> 1)) - len(normalizedSuffix) {
+		panic("context size overflow")
+	}
 	newCtx := make([]interface{}, len(prefix)+len(normalizedSuffix))
EOF
@@ -140,2 +140,5 @@
normalizedSuffix := normalize(suffix)
if len(prefix) > (int(^uint(0) >> 1)) - len(normalizedSuffix) {
panic("context size overflow")
}
newCtx := make([]interface{}, len(prefix)+len(normalizedSuffix))
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
type Ctx map[string]interface{}

func (c Ctx) toArray() []interface{} {
arr := make([]interface{}, len(c)*2)

Check failure

Code scanning / CodeQL

Size computation for allocation may overflow High

This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.
This operation, which is used in an
allocation
, involves a
potentially large value
and might overflow.

Copilot Autofix AI 3 days ago

To fix the problem, we need to ensure that the size computation for the allocation does not overflow. This can be done by validating the size of c before performing the allocation. Specifically, we can check if len(c) is within a safe range before using it in the make function. If the size is too large, we should return an error or handle it appropriately.

Suggested changeset 1
erigon-lib/log/v3/logger.go

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/erigon-lib/log/v3/logger.go b/erigon-lib/log/v3/logger.go
--- a/erigon-lib/log/v3/logger.go
+++ b/erigon-lib/log/v3/logger.go
@@ -222,2 +222,5 @@
 func (c Ctx) toArray() []interface{} {
+	if len(c) > (1<<31-1)/2 {
+		panic("context size too large")
+	}
 	arr := make([]interface{}, len(c)*2)
EOF
@@ -222,2 +222,5 @@
func (c Ctx) toArray() []interface{} {
if len(c) > (1<<31-1)/2 {
panic("context size too large")
}
arr := make([]interface{}, len(c)*2)
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
Copy link

cla-bot bot commented Mar 7, 2025

We require contributors/corporates @manav2401, @VBulikov, @lupin012, @lystopad to read our Contributor License Agreement, please check the Individual CLA document/Corporate CLA document

@IvanBelyakoff IvanBelyakoff changed the title Merge/erigon 2.61.2 to 2.61 fix ipa Merge/erigon 2.61.2 to fork Mar 10, 2025
@IvanBelyakoff IvanBelyakoff changed the title Merge/erigon 2.61.2 to fork Upstream: Merge/erigon 2.61.2 to fork Mar 10, 2025
@IvanBelyakoff IvanBelyakoff changed the title Upstream: Merge/erigon 2.61.2 to fork Upstream: merge 2.61.2 to fork Mar 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.