Release 4.16.0

Release 4.16.0 includes multiple important features, improvements, bug fixes and some dependencies CVE fixes.

Due to this version has one critical regression in the BookKeeper client, Apache BookKeeper users are encouraged
to skip this version and upgrade to 4.16.1.

The technical details of this release are summarized below.

Breaking Changes

• Change the API for org.apache.bookkeeper.stats.Counter PR #3501
  • Change name : Counter.add --> Counter.addCount
  • Add new method Counter.addLatency to count the time and convert the time to milliseconds
• When using V2 protocol, the bookkeeper_server_ADD_ENTRY_REQUEST and bookkeeper_server_READ_ENTRY_REQUEST stats do not work, and you can use bookkeeper_server_ADD_ENTRY and bookkeeper_server_READ_ENTRY instead. PR #3837

Features

• Add Direct IO support for ledger, BP-47 (PR #3189, PR #3910, PR #3917, PR #3253, PR #3256, PR #3263, PR #3366)
• Support Intel PMem disk as journal PR #3194
• Auto Recovery support repair not adhering placement policy ledgers PR #3359
• Support setting separate ledger index directories instead of keep the same directory with ledger data Issue #3419

Improvement

• Allow to use IO uring instead of epoll PR #3595
• Fixed the pivot selection in the group quick-sort PR #3800
• Improvements in ArrayGroupSort PR #3807
• Added BatchedArrayBlockingQueue PR #3838
• Group and flush add-responses after journal sync PR #3837
• Use JNI directly for posix_fadvise PR #3824
• Improved efficiency in DigestManager.verify() PR #3810
• Made PendingAddOp thread safe PR #3784
• Single buffer for small add requests PR #3783
• Optimize ReadResponse for small entry sizes PR #3597
• Avoid extra buffer to prepend frame size PR #3560
• Bring back deleteRange for RocksDB to improve location delete performance PR #3653
• Avoid thread-local state when computing CRCs PR #3811
• Make read entry request recyclable PR #3842
• Use SingleThreadExecutor for OrderedExecutor and drainTo() tasks into local array PR #3546
• Consolidate Netty channel flushes to mitigate syscall overhead PR #3383
• Refactor Auditor to simplify the readability PR #3637
• Reduce unnecessary loop in removeIf if map is empty PR #3512
• Change order of doGcLedgers and extractMetaFromEntryLogs PR #3869
• Upgrade RocksDB version to 7.9.2 PR #3795
• Prioritize compaction of entry logs with the lowest amount of remaining usable data PR #3390
• Added flag to control whether to transition to read-only mode when any disks full PR #3212
• Ledger replicate supports throttle PR #2778
• Apply the backpressure changes on the V2 requests PR #3324
• Add small files check in garbage collection PR #3631
• Add get cluster_info REST API PR #3710
• Add new api resumeCompaction and suspendCompaction PR #3509
• Add trigger entry location index rocksDB compact REST API PR #3802
• Add Http-service to check bookie sanity state PR #3630

Notable fixes

Bookie

• Fix memory leak issue of reading small entries PR #3844
• Fix memory leak when the Bookie is in read only mode PR #3746
• Fix memory leak when closeRecovered,failed on clearing newEnsemblesFromRecovery PR #3672
• Fix memory leak when operating ledger metadata PR #3662
• Fix ByteBuf memory leak problem when setExplicitLac PR #3577
• Fix memory leak when reading entry but the connection disconnected. PR #3528
• Fix the readResponse byteBuf potential memory leak problem. PR #3525
• Modify incorrect rocksDB config level_compaction_dynamic_level_bytes to CFOptions PR #3860
• Optimize masterKeyCache StorageNotificationListener PR #3736
• Fix RegionAwareEnsemblePlacementPolicy.newEnsemble sometimes failed problem. PR #3725
• New ensemble choose different rack first. PR #3721
• Fix RegionAwareEnsemblePlacementPolicy#onBookieRackChange didn't update perRegionPlacement. PR #3666
• When call openLedgerOp, make the timeout ex is a separate error code PR #3562
• Using a separate thread pool to execute openWithMetadata PR #3548
• Change masterKeyCache to dynamic size PR #3522
• Fix group ForceWrite not take effect with forceWriteMarkerSent in while loop of ForceWriteThread PR #3454
• WriteLacResponse should be processed in the same thread as other requests PR #3452
• Update default value of allocatorPoolingConcurrency PR #3001
• Fix the infinite waiting for shutdown due to throttler limit PR #2942

Client

• LedgerHandle: do not complete metadata operation on the ZookKeeper/Metadata callback thread PR #3516
• LedgerOpenOp: Do not call blocking close() in the callback PR #3513
• Rename success with writableResult and update final writableResult about wait writeSet PR #3505
• Fix the V2 AddRequest object leak issue PR #3323
• Fix the PendingAddOp is not recycled when LedgerHandler closed PR #3321
• Make sure the LedgerHandle close callback can be completed when encounter exception PR #2913
• Change PCBC limitStatsLogging default value to true PR #3719

AutoRecovery

• Fix issue where checkAllLedgers could get stuck when read throttling is enabled PR #3655
• Shut down ReplicationWorker and Auditor on non-recoverable ZK error PR #3374
• Fix autoRecovery memory leak. PR #3361
• Fix autoRecovery does not process under-replicated empty ledgers PR #3239
• Fix bug where checkAllLedgers gets stuck when read throttling is enabled PR #3214

Others

• Support build in the arch64 linux platform PR #3828
• Support update ledger metadata option bk-cli PR #3821
• Make jvm_memory_direct_bytes_used metrics compatible with jdk8. PR #3677
• Speed up the rebuilding of RocksDB index PR #3458
• Allow run LocalBookkeeper directly in bookkeeper-server module or on IDE PR #3255

Dependency changes

Upgraded notable dependencies and address CVEs, including:

• netty
• RocksDB
• snakeyaml
• zookeeper
• Docker base image
• jackson
• protobuf
• hadoop
• vertx
• log4j2
• Jetty
• groovy

Details

https://github.com/apache/bookkeeper/pulls?q=is%3Apr+milestone%3A4.16.0+is%3Aclosed

Release 4.15.4

Release 4.15.4 includes multiple bug fixes and some dependencies CVE fixes.

Apache BookKeeper users are encouraged to upgrade to 4.15.4.
The technical details of this release are summarized below.

Highlights

Bugs

• Fix QueueEntry recycle problem PR #3747
• Fix memory leak when the Bookie is in read only mode PR #3746
• Fix RegionAwareEnsemblePlacementPolicy.newEnsemble sometimes failed problem PR #3725
• Not wrap IOException twice form checkpoint PR #3683
• Make jvm_memory_direct_bytes_used metrics compatible with jdk8 PR #3677
• Fix memory leak when closeRecovered,failed on clearing newEnsemblesFromRecovery PR #3672
• Fix RegionAwareEnsemblePlacementPolicy update rack info problem PR #3666
• Exit bookkeeper shell correctly even if fails to run for some reason PR #3663
• Fix memory leak when operating ledger metadata PR #3662
• Check client if closed when complete callback PR #3661
• Fix issue where checkAllLedgers could get stuck when read throttling is enabled PR #3655
• Fix GetBookieInfo failed event stats PR #3622
• Apply recycle logic during add entry creation but ledger close to LedgerHandleAdv PR #3621
• BookieImpl remove wait until journal quits PR #3603

Improvements

• Support build in the aarch64 linux platform PR #3828
• Use ReferenceCountUtil.release() instead of ReferenceCountUtil.safeRelease() PR #3797
• Added api/v1/bookie/cluster_info REST API PR #3713
• Expose registrationClient in DefaultBookieAddressResolver PR #3724
• New ensemble choose different rack first PR #3721
• Improve bk_server.conf docs PR #3715
• Show result of isFenced in log PR #3678
• Include bkperf into bk all package PR #3632
• Add journal file path that caused failure in multi-journal config PR #3623
• Avoid extra buffer to prepend frame size PR #3560
• Using a separate thread pool to execute openWithMetadata PR #3548
• LedgerHandle: do not complete metadata operation on the ZookKeeper/Metadata callback thread PR #3516
• ledgerFragment check and results keep order PR #3504
• Simplified No network topology script is found default log stack output PR #3496
• Support apple m1 build PR #3175

Dependency updates

• Bump vertx-web from 4.3.2 to 4.3.8 PR #3775
• Upgrade docker image version to fix CVEs PR #3640
• Bump jcommander from 1.78 to 1.82 PR #3476

Details

https://github.com/apache/bookkeeper/pulls?page=1&q=is%3Apr+label%3Arelease%2F4.15.4+is%3Aclosed

Release 4.14.7

Release 4.14.7 includes multiple bug fixes.

Apache BookKeeper users are encouraged to upgrade to 4.14.7.
The technical details of this release are summarized below.

Highlights

Bugs

• Revert PR#3653 and make delete entries batch size configurable PR #3768
• Fix memory leak when the Bookie is in read only mode PR #3746
• Fix QueueEntry recycle problem. PR #3747
• Expose registrationClient in DefaultBookieAddressResolver. PR #3724
• Added api/v1/bookie/cluster_info REST API PR #3714
• Check if channel closed before processing read request PR #3486
• Add missed call to onReadRequestFinish() when read request rejected PR #3482
• Avoid extra buffer to prepend frame size PR #3560

Details

https://github.com/apache/bookkeeper/pulls?q=is%3Apr+label%3Arelease%2F4.14.7+is%3Aclosed

Release 4.14.6

Release 4.14.6 includes multiple bug fixes and some dependencies CVE fixes.

Apache BookKeeper users are encouraged to upgrade to 4.14.6.
The technical details of this release are summarized below.

Highlights

Bugs

• Fix memory leak when reading entry but the connection disconnected. PR #3528
• When call openLedgerOp, make the timeout ex is a separate error code PR #3562
• Apply recycle logic during add entry creation but ledger close to LedgerHandleAdv PR #3621
• Fix autoRecovery memory leak. PR #3361
• Fix potential memory leak. PR #3530
• Fix the V2 AddRequest object leak issue PR #3323
• Fix ByteBuf memory leak problem when setExplicitLac PR #3617
• Fix the problem that the abnormal file causes the bookie GC fail PR #3611
• Fix the deadlock when only using io thread to handle request PR #3480
• Fix memory leak when closeRecovered,failed on clearing newEnsemblesFromRecovery PR #3672
• Fix memory leak when operating ledger metadata PR #3662
• LedgerHandle: do not complete metadata operation on the ZookKeeper/Metadata callback thread PR #3516
• Ledger replicate supports throttle PR #2778
• CheckAllLedgers in Auditor supports read throttle PR #2973
• Rename success with writableResult and update final writableResult about wait writeSet PR #3505
• LedgerFragment check and results keep order PR #3504
• Improve the throttle function PR #2991
• Fix close ledgerAuditorManager repeatedly PR #3503
• AutoRecovery - Do not call shutdown() on the main ZookKeeper client thread PR #3487
• WriteLacResponse should be processed in the same thread as other requests for the same ledgerId PR #3452
• Shutdown ReplicationWorker and Auditor on non-recoverable ZK error PR #3374
• Enhance future sync wait. PR #3336
• Pre break loop when self create layoutZNode succeed. PR #3335
• Fix the PendingAddOp is not recycled when LedgerHandler closed PR #3321
• Fix autorecovery does not process underreplicated empty ledgers PR #3239
• Fix RegionAwareEnsemblePlacementPolicy update rack info problem. PR #3666
• Fix bug where checkAllLedgers gets stuck when read throttling is enabled PR #3214
• Fix checkAllLedgersDuration compute PR #2970
• Zk client config update and bugfix for ZKMetadataClientDriver PR #2958
• Make sure the LedgerHandle close callback can be completed when encounter exception PR #2913
• Make jvm_memory_direct_bytes_used metrics compatible with jdk8. PR #3677
• Reorder the sequence of the bookkeeper server shutdown so that there are no read/ write ops while shutting down the bookie PR #2888
• Fix readlogmetadata failed bug and export entrylog file usage to output PR #2349
• MinorCompactionInterval should be greater than gcWaitTime PR #2116
• Skipping placementPolicyCheck when ledger replication disabled PR #3561
• Skip replicasCheck when replication disabled PR #3563
• Fix GetBookieInfo failed event stats PR #3622
• Add journal file path that caused failure in multi-journal config PR #3634
• Include bkperf into bk all package PR #3632
• Fix maven javadoc generate issues PR #3615
• Switch to rely on SslEngine for Hostname Verification PR #3310
• Simplified No network topology script is found default log stack output PR #3496
• Fix jvm_memory_direct_bytes_used metrics when using jdk11+ PR #3252
• Fix the 3144 revert issue PR #3283
• Avoid init WriteSet when waitForWriteSetMs < 0. PR #3325
• Tuning PendingReadOp.java seq PR #3330
• Optimize log for failed to write entry PR #3463
• Reduce unnecessary loop in removeIf if map is empty PR #3512
• Deduplicate error log for SSLException PR #3320
• Fix underReplicatedLedgerTotalSize calculate problem. PR #3337
• Fix wrong ledger id parse radix for index relocation file in IndexPersistenceMgr PR #2944
• Fix bookie CI test not run PR #3367
• Update doc about flushInterval config PR #3601
• Show content of map PR #3538
• Add example for multiple server list in metadataServiceUri configuration PR #3580
• Replace sleep with await to avoid flaky test in SlowBookieTest PR #3581
• Add javadoc comments for test classes PR #3587
• Correct link class name PR #3594
• Fix typo in bk_server.conf PR #3574
• Fix flaky test testAutoRecoverySessionLoss PR #3576
• Fix flaky tests in AuditorReplicasCheckTest PR #3551
• Fix the tls failed test PR #3448
• Backport BookieBackpressureForV2Test to branch-4.14 PR #3443
• Fix Flaky-test: testBookieContinueWritingIfMultipleLedgersPresent PR #3421
• BookieAutoRecoveryTest.testEmptyLedgerLosesQuorumEventually fix flaky test, ensure that the Auditor is alive PR #3149
• Close journal channel in testJunkEndedJournal PR #3307
• Fix stream storage flaky tests, statelib test times out PR #2883
• Fix flaky AutoRecoveryMainTest PR #2881
• Support apple m1 build PR #3175
• Refactor ByteBuf release method in module distributedlog-core/distributedlog-protocol PR #3693
• Refactor ByteBuf release method in module distributedlog-core PR #3691
• Refactor ByteBuf release method in tools PR #3687
• Refactor ByteBuf release method in stream/statelib PR #3689
• Refactor ByteBuf release method in DefaultEntryLogger PR #3673
• Refactor ByteBuf release method in InterleavedLedgerStorage PR #3674

Improvements

• Bring back deleteRange for RocksDB to improve location delete performance PR #3653
• Consolidate Netty channel flushes to mitigate syscall overhead PR #3383
• Using a separate thread pool to execute openWithMetadata PR #3548
• Replace unsafe NoEntryException with IOException PR #2909
• Update default value of allocatorPoolingConcurrency PR #3001
• Apply the backpressure changes on the V2 requests PR #3324
• LedgerOpenOp: Do not call blocking close() in the callback PR #3513

Metrics changes

• Add rocksDB read latency and read from storage latency for entry reading PR #3647
• Add latency s...

Release 4.15.3

Release 4.15.3 includes multiple bug fixes and some dependencies CVE fixes.

Apache BookKeeper users are encouraged to upgrade to 4.15.3.
The technical details of this release are summarized below.

Notice

• 4.15.0 introduced a breaking change in the RocksDB configuration in 4.15.0. Release 4.15.3 allows users to safely upgrade from 4.14.x without losing the RocksDB runtime configuration
  For more details, refer to https://lists.apache.org/thread/drh4p5prxbcs8gszhxnd1xsv0g48vvbt
  See PR #3523

• Timeout exceptions are now handled in a better way during reads.
  See PR #3562

Highlights

Dependency updates

• Upgrade dependencies for CVE-2022-3171 and CVE-2022-42003. PR #3579
• Bump jackson version to 2.13.4 . PR #3518

Bugs

• Fix ByteBuf memory leak problem when setExplicitLac. PR #3557
• Flush time started moved to after lock. PR #3570
• Skip replicasCheck when replication disabled. PR #3563
• Skipping placementPolicyCheck when ledger replication disabled. PR #3561
• Fix the deadlock when only using io thread to handle request. PR #3480
• Fix memory leak when reading entry but the connection disconnected. PR #3528
• Fix byteBuf potential memory leak problem. PR #3525
• LedgerOpenOp: Do not call blocking close() in the callback. PR #3513
• Rename success with writableResult and update final writableResult about wait writeSet. PR #3505
• Avoid closing the ledgerAuditorManager twice in the close method. PR #3503

Improvements

• When call openLedgerOp, make the timeout ex is a separate error code. PR #3562
• Make the rocksDB configuration compatible with previous versions. PR #3523
• Reduce unnecessary loop in removeIf if map is empty. PR #3512
• Update default value of allocatorPoolingConcurrency. PR #3001

Release 4.15.2

Highlights

Dependency updates

• Bump snakeyaml from 1.31 to 1.32 to solve CVE-2022-38752. PR #3491
• upgrade hadoop version to 3.2.4. PR #3485
• [security] Upgrade Jetty to 9.4.48.v20220622 to get rid of CVE-2022-2047. PR #3404
• Bump dependency check to 7.1.2 to avoid FP. PR #3470
• Bump snakeyaml from 1.30 to 1.31 to solve CVE-2022-25857. PR #3469

Bugs

• AutoRecovery - Do not call shutdown() on the main ZookKeeper client thread. PR #3487
• Check if channel closed before processing read request. PR #3486
• Add missed call to onReadRequestFinish() when read request rejected. PR #3482
• Fix the deadlock when only using io thread to handle request. PR #3480

Metrics changes

• Add latency stats for entry location index lookup so that possible RocksDB bottleneck can be detected. PR #3444

Release 4.15.1

Highlights

Bugs

• Fix AutoRecovery memory leak. PR #3361
• Fix: NPE in RackawareEnsemblePlacementPolicyImpl logged by AutoRecovery. PR #3350
• Fix underReplicatedLedgerTotalSize calculate problem. PR #3337
• Fix JVM exited when running localbookie with jdk17. PR #3334
• Fix the V2 AddRequest object leak issue. PR #3323
• Fix the PendingAddOp is not recycled when LedgerHandler closed. PR #3321
• Bookie can't start after rebooting due the cookie mistmatch. PR #3308
• Close journal channel in testJunkEndedJournal. PR #3307
• Fix jvm_memory_direct_bytes_used metrics when using jdk11+. PR #3252
• AutoRecovery does not process underreplicated empty ledgers. PR #3239
• Fix wrong ledger id parse radix for index relocation file in IndexPersistenceMgr. PR #2944
• Fix the infinite waiting for shutdown due to throttler limit. PR #2942
• Make sure the LedgerHandle close callback can be completed when encounter exception. PR #2913

Improvements

• If ensembleList is empty, return PlacementPolicyAdherence.FAIL. PR #3369
• reduce unnecessary checkpoints. PR #3341
• Pre break loop when self create layoutZNode succeed. PR #3335
• Avoid init WriteSet when waitForWriteSetMs < 0. PR #3325
• Tuning PendingReadOp.java seq. PR #3330
• Apply the backpressure changes on the V2 requests. PR #3324
• Deduplicate error log for SSLException. PR #3320
• Switch to rely on SslEngine for Hostname Verification. PR #3310
• Try to use Java9 CRC32C when JNI based CRC is not available. PR #3309
• Enhance future sync wait. PR #3336
• validate diskUsageThreshold and diskUsageLwmThreshold. PR #3285
• Replace unsafe NoEntryException with IOException. PR #2909
• let bookie quit if journal thread exit. PR #2887
• Ledger replicate supports throttle. PR #2778
• minorCompactionInterval should be greater than gcWaitTime. PR #2116
• Optimize concurrent collection's shrink logic. PR #3417
• Make BookieFileChannel interface public. PR #3396
• Prioritize compaction of entry logs with the lowest amount of remaining usable data. PR #3390
• Switch back ordered executor to LinkedBlockingQueue. PR #3384
• Consolidate Netty channel flushes to mitigate syscall overhead. PR #3383
• Shut down ReplicationWorker and Auditor on non-recoverable ZK error. PR #3374
• BP-41 Add flag to enable/disable BookieAddressResolver. PR #3356
• release the bookie from QuarantinedBookies when health check is disabled. PR #3349

Metrics changes

• add journalCbQueueLatency. PR #3364
• add writeThreadQueuedLatency. PR #3363
• add metric cbThreadPoolQueueSize. PR #3424

Dependency updates

• Upgrade log4j2 to 2.18.0. PR #3434
• upgrade groovy from 2.5.17 to 3.0.11 to fix CVE-2019-11358(7.5). PR #3346

Details

https://github.com/apache/bookkeeper/pulls?q=is%3Apr+label%3Arelease%2F4.15.1+is%3Aclosed+is%3Amerged

Release 4.14.5

Release notes: https://bookkeeper.apache.org/release-notes#4145

Release 4.15.0

Apache Bookkeeper release 4.15.0

Release 4.15 includes many upgrades to third party libraries marked with CVEs, adds more configuration options, extends REST API, adds an option to run without journal, improves memory utilization and stability, and more!

Apache BookKeeper users are encouraged to upgrade to 4.15.0. The technical details of this release are summarized below.

Breaking Changes

• BookieServer API changed and the code that creates its instances will require addition of the UncleanShutdownDetection parameter. See PR 2936 for details and examples.

• Bookie class now is an interface with implementation in BookieImpl. Code that uses it may need changes. For details please refer to PR 2717.

• LedgerUnderreplicationManager interface added a new method. Code that implements the interface will need changes. See PR 2805 for details.

• MetadataBookieDriver interface added a new method and removed an old one. RegistrationManager interface added a new method. ByteBufAllocatorWithOomHandler interface is added and used instead of the ByteBufAllocator in multiple places. Code that implements the interfaces will need changes. See PR 2901 for details.

Highlights

Configuration

• ledgerMetadataVersion: BookKeeper-Client config to write ledger metadata with configured version.
• clientTcpUserTimeoutMillis: Added TCP_USER_TIMEOUT to Epoll channel config.
• auditorMaxNumberOfConcurrentOpenLedgerOperations and auditorAcquireConcurrentOpenLedgerOperationsTimeOutMSec Add auditor get ledger throttle to avoid auto recovery zk session.
• dbStorage_rocksDB_format_version make rocksdb format version configurable.

Features

• Running without journal. See BP-46 for details.
• A REST API to get or update bookie readOnly state. Details
• Separate config files for Rocks DB. Details

Improvements

• Build and tests work on JDK 17
• CLI: listunderreplicated command has an option to return count without printing all ledgers #3228
• Stream Storage: support an optional time to live (TTL) on a per table basis #2775
• Added dDb ledger index rebuild operation and CLI commands #2774
• Support multi ledger directories for rocksdb backend entryMetadataMap #2965
• Improved memory utilization
  • support shrink for ConcurrentLong map or set #3074
  • reduce unnecessary expansions for ConcurrentLong map and set #3072
• Read speed rate limiter for scanning entry log file in entryMetadataMap rebuild #2963
• Other improvements in areas such as test, documentation, CI, metrics, logging, and CLI tools.

Notable fixes

• Bookkeeper client might not close the channel for several minutes after a Bookie crashes #2482
• Stream storage: Ensure progress while restoring from checkpoint. #2764
• Entry Log GC may get blocked when using entryLogPerLedgerEnabled option #2779
• Fix region aware placement policy use disk weight #2981
• Some cases that could cause RocksDB segfault
• DistributedLogManager can skip over a segment on read. #3064
• Backpressure: check all bookies of writeset are writable #3055
• Fix Journal.ForceWriteThread.forceWriteRequests.put deadlock #2962
• PendingReadOp: Fix ledgerEntryImpl reuse problem #3110
• Region/rack aware placement policy: replace bookie bug #2642
• ReplicationWorker: numLedgersReplicated metric does not update #3218
• Force GC doesn't work under forceAllowCompaction when disk is full #3205

Dependencies changes

Upgraded dependencies to address CVEs include:

• vertx
• freebuilder
• libthrift
• netty
• bouncycastle
• commonsIO
• jetty
• log4j
• grpc
• protobuf
• snakeyaml
• RocksDB
• jackson
• jackson-databind
• Zookeeper
• http-core
• dropwizard metrics

Dependency on log4j v.1 is removed.

Details

https://github.com/apache/bookkeeper/issues?q=+label%3Arelease%2F4.15.0

Release 4.14.3

release-4.14.3

release-4.14.3