v0.7.0-rc1 - Stable `Arabica`

This pre-release brings a lot of breaking changes, including an upgrade to a breaking version of celestia-app as well as the addition of versioning to our pubsub topics, meaning we have also upgraded the chain ID of arabica to arabica-4.

The state from the previous arabica chain has been preserved, so it is possible to migrate keys to any already-funded accounts to the node's keystore (by default, located at ~/.celestia-<node_type>-arabica-4/keys/keyring-test/).

Note: please re-initialise the node's store.

Most notably, however, this pre-release includes a fix (#1639 and #1684) that address instability issues previously experienced on both mocha and arabica when attempting to bootstrap onto the networks. Special thanks to our master debugger @Wondertan for finding the issue and to @vgonkivs for implementing the fix.

What's Changed

BREAKING

• refactor!: naming PFD/PayForData to PayForBlob by @distractedm1nd in #1582
• chore!: bump celestia app v0.12.0-rc2 by @evan-forbes in #1617
• chore!: bump arabica chain-id and genesis hash by @Bidon15 in #1645
• chore!: upgrade to celestia-app 0.12.0-rc4 by @rootulp in #1648
• chore!: bump celestia-app to v0.12.0-rc5 by @evan-forbes in #1679
• feat!: add versioning for pubsub topics in fraud, header packages by @vgonkivs in #1620
• chore(nodebuilder/p2p)!: Upgrade arabica chain ID to arabica-4 by @renaynay in #1696
• feat!: Move Dockerfile to the root of the project by @jrmanes in #1693
• chore: bump celestia-app v0.12.0-rc6 by @evan-forbes in #1689
• chore!: bump arabica-4 genesis hash by @Bidon15 in #1701

FEATURES

• feat(share/discovery): Add callback on peers set updates. by @walldiss in #1609
• feat(share/eds): adding traces to edsstore, write/read eds, and getters by @distractedm1nd in #1594
• feat(share): implements sync CascadeGetter by @Wondertan in #1628
• feat: using tmjson encoding on core.Header by @distractedm1nd in #1641
• feat(node | das | libs/header/sync): add total uptime node metrics + totalSampled das metrics + totalSynced sync metrics by @derrandz in #1638

IMPROVEMENTS

• improvement(das): Add timeout for sampling in DASer by @walldiss in #1615
• refactor(share/p2p): ironing out differences between shrex protocols by @distractedm1nd in #1611
• refactor(header): Extract CoreExchange and CoreListener out of the header root package by @renaynay in #1627
• refactor(ipld): removing nmt hasher by @distractedm1nd in #1651

BUG FIXES

• fix(header/p2p): fix error string by @vgonkivs in #1618
• fix(daser) replace catchUpDoneCh channel gracefuly by @walldiss in #1631
• fix(p2p/pubsub): extend pubsub options by @vgonkivs in #1634
• fix(core/listener): Only broadcast datahash if core node is not syncing by @renaynay in #1652
• fix(share/discovery): decouple peer discovery from event processing by @vgonkivs in #1639
• fix(libs/header/sync): Make ranges.Add thread-safe by @tzdybal in #1649
• fix: namespaced_data endpoint by @rootulp in #1526
• fix(header/p2p): add trivial retrying for trustedPeer requested by @Wondertan in #1647
• fix(share/discovery): add loop to read events from the channel multiple times by @vgonkivs in #1684
• bugfix(libs/header): fix data race in syncer by @vgonkivs in #1682

MISC

• chore(nodebuilder/p2p): Add Kaarina to arabica by @renaynay in #1619
• docs: fix minor typos by @omahs in #1636
• chore(swamp): replace cosmos prefix with celestia in swamp tests by @vgonkivs in #1583
• chore(nodebuilder/p2p): Update Arabica kaarina peerID by @Bidon15 in #1650
• nodebuilder: make proxy constructors private in all modules by @jpserrat in #1643
• fix(Makefile): make install should not invoke build by @Wondertan in #1673
• chore(deps): bump golangci/golangci-lint-action from 3.3.1 to 3.4.0 by @dependabot in #1655
• bump(metrics): upgrade otel to the latest version by @vgonkivs in #1537
• fix: flake in TestNamespaceHasher_CorruptedData by @distractedm1nd in #1686
• dep(pubsub): upgrade pubsub version by @vgonkivs in #1688
• chore(go.mod): Depend on upstream libp2p (AKA remove hlibp2p) by @renaynay in #1700

New Contributors

• @omahs made their first contribution in #1636
• @jpserrat made their first contribution in #1643
• @jrmanes made their first contribution in #1693

Full Changelog: v0.6.4...v0.7.0-rc1

v0.6.4 - Stable Mocha

Another patch release that fixes the DataRoot computation issue for when evidence is created on the core network and bugs when connection events are not read out correctly.

BUG FIXES

• fix(share/discovery): add loop to read events from the channel multiple times #1684
• fix(header): always calculate the data root even for blocks with no txs #1678

v0.6.3 - Stable Mocha

This patch release notably includes some fixes related to stabilising mocha.

Note: this release includes a config change, so nodes running on mocha will need to re-initialise their configs.

This release is not compatible with arabica.

BREAKING

• feat(header)!: introduce header.Header interface (#1304) by @tzdybal
• feat(nodebuilder/p2p)!: bump arabica chain-id and genesis hash (#1645) by @Bidon15

BUG FIXES

• fix(daser) replace catchUpDoneCh channel gracefuly (#1631) by @walldiss
• fix(share/discovery): decouple peer discovery from event processing (#1639) by @vgonkivs
• fix(libs/header/sync): Make ranges.Add thread-safe (#1649) by @tzdybal
• fix(header/p2p): add trivial retrying for trustedPeer requested (#1647) by @Wondertan '

FEATURES

• feat(share): implements sync CascadeGetter (#1628) by @Wondertan

IMPROVEMENTS

• refactor(share/p2p): ironing out differences between shrex protocols (#1611) by @distractedm1nd
• refactor(header): Extract CoreExchange and CoreListener out of the header root package (#1627) by @renaynay
• nodebuilder: make proxy constructors private in all modules (#1643) by @jpserrat

MISC

• docs: fix minor typos (#1636) by @omahs

Full Changelog: v0.6.2...v0.6.3

v0.6.2

What's Changed

In terms of features, implementing auth middleware on the RPC server as well as adding authentication to the RPC client are the biggest new feature introductions. Documentation on how to use an authenticated RPC client will come shortly. For now, using client.NewPublicClient will allow access to public endpoints.

This release also includes several improvements and bug fixes, most notably upgrading libp2p dependencies across the board, as well as several feature introductions and refactorings that progress us towards our goal of refactoring our blocksync architecture.

Two important notes:

1. We recommend re-initialising your node for this release as we introduced some new config parameters.
2. This release does not fix stability issues with Arabica testnet bootstrappers - we are still in the process of investigating.

BREAKING

• fix(node/p2p)!: remove resource manager and it's API by @Wondertan in #1500
• improvement(header|p2p)!: remove limit status code by @vgonkivs in #1530
• refactor(nodebuilder/p2p): Add errors for all methods by @renaynay in #1539
• Revert "fix(node/p2p)!: remove resource manager and it's API (#1500)" by @Wondertan in #1570
• feat(header|syncer)!: add params to syncer by @vgonkivs in #1544
• feat!: share.Getter and getters.IPLDGetter by @distractedm1nd in #1518

FEATURES

• feat(api/rpc): Implement auth middleware on Server by @renaynay in #1402
• feat(share): Ensure empty CAR exists in eds.Store by @distractedm1nd in #1409
• feat(shrex/eds): Client/Server by @distractedm1nd in #1431
• feat(api/client): Implement RPC client authentication by @renaynay in #1426
• feat(header|p2p): add blacklisting to peerTracker by @vgonkivs in #1532
• feat(eds/store): store.GetDAH by @distractedm1nd in #1511
• feat(cmd/celestia): Add auth subcommand to bridge, full and light by @renaynay in #1429
• feat(share/eds): Create a blockservice.BlockGetter adapter for eds.Store by @walldiss in #1561
• feat(share): getters.StoreGetter by @distractedm1nd in #1528
• feat(share): getters.TeeGetter by @distractedm1nd in #1540
• feat(ShrEx/ND): Create client/server for namespaced data communication by @walldiss in #1497

IMPROVEMENTS

• improvement(cel-key): Network alias check for --p2p.network flag by @renaynay in #1482
• refactor: exporting nodebuilder/header.InitStore, moving mock to nodebuilder/header by @distractedm1nd in #1515
• feat(header/p2p): implement retry mechanism for header/p2p session by @vgonkivs in #1477
• deps: bump go-libp2p and all supporting pkgs by @Wondertan in #1547
• improvement(header/p2p): change store to getter inside headerEx server by @vgonkivs in #1604
• share/sharexpush: decrease ttl for seen messages in pubsub by @vgonkivs in #1580
• improvement(header/p2p): Implement context deadline on requests to store inside server by @renaynay in #1588

BUG FIXES

• fix: return a struct instead of a pointer in params creation by @vgonkivs in #1507
• fix(eds): WriteEDS thread safety for concurrent writingSessions by @distractedm1nd in #1498
• fix(header/p2p): preconnect trusted peers on header-ex start by @Wondertan in #1552
• fix(ipld): Make namespaceHasher fully implement hash.Hash and update go-multihash by @Wondertan in #1546
• fix(share): order proof nodes by depth before returning by @walldiss in #1545
• fix: use udp/quic instead of tcp/quic in NoAnnounceAddress by @tzdybal in #1560
• fix(header): don't stop Subscriber in Syncer and remove Stop from interface by @Wondertan in #1554
• fix(header/p2p): fix crash in server by @vgonkivs in #1569
• fix(cel-key): keyring-dir flag should override node.type if provided. by @renaynay in #1571
• bugfix(header/p2p): handle long live context in head request by @vgonkivs in #1587
• fix(cel-key): update flag name to --p2p.network by @kevinji in #1590
• fix(fraud): fix race by using correct peer id in log statement by @Wondertan in #1605
• fix(header/store): cleanup subscription early if was cancelled by @Wondertan in #1606
• fix(header/p2p): add peerScore decreasing in case of failed request by @vgonkivs in #1603

MISCELLANEOUS

• chore(all): lint all the comment with cfmt and add cfmt to lint rules by @Wondertan in #1501
• fix: correct spacing of celestia banner by @tzdybal in #1505
• github/CI: Fix over kill of '.gitignore'. by @HoytRen in #1446
• share/p2p: move eds sub by @vgonkivs in #1519
• chore(header/p2p): improve logging by @vgonkivs in #1520
• chore(nodebuilder/p2p): Update limani nodekey for arabica by @renaynay in #1523
• improvement(header|p2p): log the reason on blocking by @vgonkivs in #1534
• refactor(eds/store): store_test.go cleanup by @distractedm1nd in #1538
• chore(modheader): don't ignore Syncer error on start by @Wondertan in #1553
• chore(deps): bump mheap/github-action-required-labels from 2 to 3 by @dependabot in #1559
• tests(swamp): correct fraud syncing test by @Wondertan in #1555
• test(core|state|swamp): unify and refactor core/app testing utilities by @Wondertan in #1551
• chore(deps): bump golang.org/x/crypto from 0.3.0 to 0.5.0 by @dependabot in #1577
• chore(deps): bump github.com/ipfs/go-blockservice from 0.4.0 to 0.5.0 by @dependabot in #1574
• chore(deps): bump golang.org/x/text from 0.5.0 to 0.6.0 by @dependabot in #1572
• chore(deps): bump go.uber.org/multierr from 1.8.0 to 1.9.0 by @dependabot in #1573
• chore(deps): bump github.com/ipfs/go-ipld-cbor from 0.0.5 to 0.0.6 by @dependabot in #1468
• docker: Update entrypoint.sh for multiple networks by @jcstein in #1522
• chore: updating rsmt2d to v0.8.0 and removing test skip by @distractedm1nd in #1585
• refactor(shrexeds): removing retry logic from client by @distractedm1nd in #1591
• chore(deps): bump google.golang.org/grpc from 1.51.0 to 1.52.0 by @dependabot in #1600
• chore(deps): bump github.com/libp2p/go-libp2p-routing-helpers from 0.4.0 to 0.6.0 by @dependabot in #1601
• chore(header/p2p): improve logging and error wrapping by @Wondertan in #1608
• feat(header/p2p): add traces to server by @vgonkivs in #1589

New Contributors

• @jcstein made their first contribution in #1522
• @kevinji made their first contribution in #1590

Full Changelog: v0.6.1...v0.6.2

v0.6.2-rc1

This patch release candidate contains bug fixes found during debugging arabica testnet as well as the implementation of authorization for RPC.

What's Changed

BREAKING

• fix(node/p2p)!: remove resource manager and it's API by @Wondertan in #1500
• improvement(header|p2p)!: remove limit status code by @vgonkivs in #1530
• refactor(nodebuilder/p2p)!: Add errors for all methods by @renaynay in #1539

FEATURE

• feat(api/rpc): Implement auth middleware on Server by @renaynay in #1402
• feat(shrex/eds): Client/Server by @distractedm1nd in #1431
• feat(header|p2p): add blacklisting to peerTracker by @vgonkivs in #1532
• feat(eds/store): store.GetDAH by @distractedm1nd in #1511
• feat(cmd/celestia): Add auth subcommand to bridge, full and light by @renaynay in #1429

IMPROVEMENT

• improvement(cel-key): Network alias check for --p2p.network flag by @renaynay in #1482
• feat(header/p2p)!: implement retry mechanism for header/p2p session by @vgonkivs in #1477

BUG FIX

• fix: return a struct instead of a pointer in params creation by @vgonkivs in #1507
• fix(eds): WriteEDS thread safety for concurrent writingSessions by @distractedm1nd in #1498

MISCELLANEOUS

• chore(all): lint all the comment with cfmt and add cfmt to lint rules by @Wondertan in #1501
• fix: correct spacing of celestia banner by @tzdybal in #1505
• github/CI: Fix over kill of '.gitignore'. by @HoytRen in #1446
• share/p2p: move eds sub by @vgonkivs in #1519
• chore(header/p2p): improve logging by @vgonkivs in #1520
• chore(nodebuilder/p2p): Update limani nodekey for arabica by @renaynay in #1523
• refactor: exporting nodebuilder/header.InitStore, moving mock to nodebuilder/header by @distractedm1nd in #1515
• improvement(header|p2p): log the reason on blocking by @vgonkivs in #1534

Full Changelog: v0.6.1...v0.6.2

v0.6.1

This release contains a configuration change for a bootstrapper's p2p resource manager. See here for more details.

What's Changed

Improvement

• feat(nodebuilder/p2p): Enable InfiniteLimits for resource manager if bootstrapper by @renaynay in #1496

MISCELLANEOUS

• chore(markdownlint): disable 'Multiple headings with the same content' rule by @Wondertan in #1480

Full Changelog: v0.6.0...v0.6.1

v0.6.0 - Mocha

This release upgrades celestia-node's dependency on celestia-app to incorporate a fix to a share splitting bug.

The binary will also now point to mocha as the default network.

Also, transaction methods have been extended to include a fee argument which breaks the public API (and REST endpoints). Please see #1484 for further details.

What's Changed

BREAKING

• chore!(nodebuilder/p2p): Upgrade mamaki to mocha by @renaynay in #1434
• refactor(nodebuilder/p2p): Switch default network from arabica to mocha by @renaynay in #1462
• feat(state)!: Adding fee parameter to CoreAccessor operations by @distractedm1nd in #1484

BUG FIXES

• fix(state): Verify the state inclusion proof without splitting keys by @evan-forbes in #1483

MISCELLANEOUS

• chore: bump celestia-app to v0.11.0 by @evan-forbes in #1478

Full Changelog: v0.5.0...v0.6.0

v0.5.0

This release contains several weeks-worth of progress in the direction of our three primary work streams: building out celestia-node's public API, hardening our header-ex, and refactoring blocksync storage (all of these epics can be viewed in more depth as they are pinned in our issues).

This release contains two breaking changes to the public API, including:

• port 26658 is default for RPC communication with the node
• port 26659 exposes a RESTful gateway to access node endpoints that were previously available on 26658

as well as a breaking change to the CLI for both the celestia and cel-key binaries:

• --node.network is now --p2p.network

and a breaking upgrade of our celestia-app dependency which changes the rsmt2d codec used and fixes an issue with share splitting.

What's Changed

BREAKING

• refactor!(params): Moving params to nodebuilder and adding config by @distractedm1nd in #1168
• feat!(api/rpc): OpenRPC scaffolding, migrating OpenAPI gateway to api/gateway by @distractedm1nd in #1175
• feat!(gateway): reenabling gateway, adding flag to enable by @distractedm1nd in #1199
• chore!: bump celestia-app to v0.10.0-rc1 by @evan-forbes in #1338
• chore!(nodebuilder/p2p): increment the chain-id for arabica by @evan-forbes in #1370
• chore!(nodebuilder/p2p): change genesis hash for upgraded arabica-2 by @Bidon15 in #1371
• refactor!(cmd/cel-key): Unify flag names --node.network -> --p2p.network by @renaynay in #1443

FEATURE

• feat(blocksync): share.WriteEDS by @distractedm1nd in #1139
• feat(blocksync): share.ReadEDS by @distractedm1nd in #1158
• feat(rpc): Add client wrapper by @distractedm1nd in #1195
• feat(rpc): tooling for openrpc spec autogeneration by @distractedm1nd in #1283
• feat(das): make WaitCatchUp public by @walldiss in #1380
• feat(module/das): add WaitCatchUp method to the module by @Wondertan in #1383
• feat(share): EDSStore scaffolding by @distractedm1nd in #1232
• feat(nodebuilder/p2p): Implement Module by @renaynay in #1285
• feat(share): Implement ODSreader by @walldiss in #1377
• feat(share): add get proof by namespace by @walldiss in #1339
• feat(header/p2p): implement GetVerifiedRangeByHeight by @vgonkivs in #1305
• feat(share): Supplying eds.Store to bridge and full nodes by @distractedm1nd in #1363
• feat(rpc): creating scaffolding for JWT authentication by @distractedm1nd in #1325
• feat(share): eds.Blockstore by @distractedm1nd in #1395
• feat(share/discovery): add Peers method by @vgonkivs in #1451
• feat(share/availability): extend SharesAvailable method with peer param by @vgonkivs in #1452
• feat(share/blocksync): implementation of ShrEx/Sub by @vgonkivs in #1436
• feat(nodebuilder/node): Implement Module by @renaynay in #1313

IMPROVEMENT

• fix: removing protocolIDs from global namespace in p2p and fraud by @distractedm1nd in #1268
• refactor(nodebuilder): Allow custom networks to work over already-initialised store by @renaynay in #1270
• feat(blocksync): Benchmarks for ReadEDS and WriteEDS by @distractedm1nd in #1197
• feat(ipld): integration test for namespace hasher by @distractedm1nd in #1256
• refactor(share): move bad encoding fraud proof to share pkg by @walldiss in #1241
• refactor(daser): use functional options pattern to configure daser by @derrandz in #1225
• fix(rpc): custom JSON ExtendedHeader marshal/unmarshaling using amino wrapper by @distractedm1nd in #1292
• feat(rpc): adding daser stub for friendly API error by @distractedm1nd in #1308
• feat(api/gateway): Implement context cancellation middleware by @renaynay in #1321
• improvement(header/server): implement timeout for GetRangeByHeight request by @vgonkivs in #1318
• fix(rpc): fraud.Proof (un)marshalling and subscription as chan by @distractedm1nd in #1307
• improvement(header/p2p): get bestHead if timeout is reached by @vgonkivs in #1319
• refactor(ipld): use Set/GetCell API from rstm2d by @Wondertan in #1173
• refactor(nodebuilder/p2p): Allow specifying network alias with --p2p.network flag by @renaynay in #1357
• tests(nodebuilder): sync and das over non-empty blocks by @Wondertan in #1341
• refactor: changing nodebuilder/<pkg>/service.go to <pkg>.go by @distractedm1nd in #1301
• improvement(header/store): make all global variables configurable by @vgonkivs in #1388
• improvement(header/p2p): parallelize GetRangeByHeight request by @vgonkivs in #1206
• improvement(header/p2p): unify sendMessage method to request headers by @vgonkivs in #1405
• feat(header/p2p): add functional params for header/p2p package by @vgonkivs in #1398
• feat(share): Periodic GC over EDSStore by @distractedm1nd in #1359
• refactor(share/eds): Store methods take DataHash instead of share.Root by @renaynay in #1439
• Extract common parsing logic into single function. by @HoytRen in #1394
• feat(header/p2p): implement gcing for peerTracker by @vgonkivs in #1298
• improvement(share/discovery): send peer to multiple readers by @vgonkivs in #1460

BUG FIX

• fix: restricting PFD metrics to successful PFDs by @distractedm1nd in #1255
• fix(das): Only debug log sampled header if it was actually successful by @renaynay in #1295
• fix(header): Extend ValidateBasic on ExtendedHeader to include data root check by @renaynay in #1364
• fix(coreaccessor): pfd submission nil check by @distractedm1nd in #1368
• fix(eds/byzantine): trim NMTWrapper's namespace during BEFP validation by @evan-forbes in #1354
• fix: allowing hostnames in gateway/rpc addr field by @distractedm1nd in #1378
• fix: pointing make openrpc-gen to the correct cmd by @distractedm1nd in #1390
• fix(header): add extra arg to fix build by @evan-forbes in #1410
• refactor(nodebuilder/p2p): Extend p2p.Module method signatures to contain context.Context as first param by @renaynay in #1415

MISCELLANEOUS

• chore: gen less acounts to reduce test execution times by @Wondertan in #1253
• chore(share/ipld): remove dead code by @Wondertan in #1257
• refactor(core/fetcher): Improve error messages for fetcher methods by @renaynay in #1263
• refactor(nodebuilder/header): Clarify error for init store failure by @renaynay in #1272
• add semver tags to docker image builds by @jbowen93 in #1277
• Makefile: update go-acc version due to errors and new release by @MSevey in #1281
• chore(nodebuilder/p2p): Update to new arabica chain-id by @evan-forbes in #1284
• refactor(share/ipld): Use NamespaceSize var for consistency by @renaynay in #1297
• chore(deps): bump golangci/golangci-lint-action from 3.2.0 to 3.3.0 by @dependabot in #1264
• chore...

v0.5.0-rc5

What's Changed

• chore!(nodebuilder/p2p): change genesis hash for upgraded arabica-2 by @Bidon15 in #1371

Full Changelog: v0.5.0-rc4...v0.5.0-rc5

v0.5.0-rc4

What's Changed

• chore(deps): bump go.opentelemetry.io/otel/metric from 0.32.1 to 0.33.0 by @dependabot in #1267
• Update Docker image build pipelines by @jbowen93 in #1337
• chore!(nodebuilder/p2p): increment the chain-id for arabica by @evan-forbes in #1370

Full Changelog: v0.5.0-rc3...v0.5.0-rc4