Apache RocketMQ Files

Compared to the previous version 5.3.3, this is a bug-fix release that includes numerous improvements to enhance stability, performance, and correctness. Key highlights include critical dependency upgrades (e.g., Netty, OpenTelemetry), significant stability enhancements like the TimerWheel snapshot mechanism for reliable recovery, and a wide range of bug fixes across modules such as Broker, Proxy, and Client. Thank you for your attention, If you have any suggestions please reply to this email.

What's Changed

• [ISSUE [#9337]] Upgrade Netty to 4.1.119.Final by @weihubeats in https://github.com/apache/rocketmq/pull/9338
• [ISSUE [#9253]] Make the message visible earlier when response process encounters errors by @redlsz in https://github.com/apache/rocketmq/pull/9255
• [ISSUE [#8651]] Fix revive topic process lag metric calculation by @redlsz in https://github.com/apache/rocketmq/pull/8652
• [ISSUE [#8629]] Add proxy RocketmqTraffic logger appender to separate traffic logs by @redlsz in https://github.com/apache/rocketmq/pull/8630
• [ISSUE [#8232]] Fix the issue that proxy remoting thread pool isolation does not take effect by @redlsz in https://github.com/apache/rocketmq/pull/8233
• [ISSUE [#9414]] Updated the Quick Start version in README to the 5.3.3 by @mxsm in https://github.com/apache/rocketmq/pull/9415
• [ISSUE [#9412]] Flush bak metrics file before deleting the original file by @GenerousMan in https://github.com/apache/rocketmq/pull/9413
• [ISSUE [#9403]] Fix the issue of metadata synchronization failure between master and slave when the topic has attributes by @RongtongJin in https://github.com/apache/rocketmq/pull/9404
• [ISSUE [#9416]] Fix batch send messages have the same message id when consumed by @qianye1001 in https://github.com/apache/rocketmq/pull/9417
• [ISSUE [#9379]] Set compact timestamp when timeStoreTable empty by @lizhimins in https://github.com/apache/rocketmq/pull/9419
• [ISSUE [#9418]] Fix ColumnFamilyHandle not close in PopConsumerRocksdbStore by @lizhimins in https://github.com/apache/rocketmq/pull/9420
• [ISSUE [#9352]] Fix wrong value in topic-group cache by @golden-yang in https://github.com/apache/rocketmq/pull/9424
• [ISSUE [#9441]] Add file length check in IndexService by @lizhimins in https://github.com/apache/rocketmq/pull/9442
• [ISSUE [#9443]] Fix add wrong value to topicGroupTable in ConsumerManager by @yx9o in https://github.com/apache/rocketmq/pull/9444
• [ISSUE [#9254]] Add CombineConsumeQueue to support CQ migration by @qianye1001 in https://github.com/apache/rocketmq/pull/9256
• [ISSUE [#9439]] Add escape for win in the method returning broker configuration by @Crazylychee in https://github.com/apache/rocketmq/pull/9440
• [ISSUE [#9451]] Optimize back pressure default value to 1024. by @GenerousMan in https://github.com/apache/rocketmq/pull/9452
• [ISSUE [#9454]] Make sure topic can be deleted when using RocksDB CQ by @qianye1001 in https://github.com/apache/rocketmq/pull/9457
• Fix IllegalStateException during app shutdown caused by premature shutdown trigger by @hepengju in https://github.com/apache/rocketmq/pull/9408
• [ISSUE [#9335]] Use MixAll.ROCKETMQ_HOME_DIR for home directory by @WJ66880 in https://github.com/apache/rocketmq/pull/9474
• [ISSUE [#9481]] Fix Issue 4050 didn't fix completely by @WJ66880 in https://github.com/apache/rocketmq/pull/9482
• [ISSUE [#9489]] Fix clean pull offset in #removeOffset by @3424672656 in https://github.com/apache/rocketmq/pull/9490
• [ISSUE [#8920]] Refactor SSL context loading process to support multiple protocols dynamic loading by @EnableAsync in https://github.com/apache/rocketmq/pull/9483
• [ISSUE [#9491]] Remove the volatile declaration for the two variables requestsWrite and requestsRead in the GroupCommitService class by @huanzhikang in https://github.com/apache/rocketmq/pull/9492
• [ISSUE [#9486]] Fix missing of valueOfCurrentMinusBorn equals checkImmunityTime by @WJ66880 in https://github.com/apache/rocketmq/pull/9487
• [ISSUE [#9497]] Fix IndexOutOfBoundsException in getEarliestMessageTime when running in IPv6 environment by @woaishixiaoxiao in https://github.com/apache/rocketmq/pull/9498
• [ISSUE [#9479]] Relax topic length constraint when creating retry and dlq topics by @Allenhuangrui in https://github.com/apache/rocketmq/pull/9480
• [ISSUE [#9476]] Set queueId to 0 in DefaultTransactionalMessageCheckListener#toMessageExtBrokerInner to improve performance by @mxsm in https://github.com/apache/rocketmq/pull/9477
• [ISSUE [#9447]] Avoiding possible resource leaks in InputStream by @AROP-REPLICATION in https://github.com/apache/rocketmq/pull/9459
• [ISSUE [#9467]] Implement paged retrieval of Topic and SubscriptionGroup information by @ltamber in https://github.com/apache/rocketmq/pull/9468
• [ISSUE [#8920]] Make TLS certificate watch interval configurable by @EnableAsync in https://github.com/apache/rocketmq/pull/9513
• fix CONTRIBUTING.md spelling mistake by @datagramsocket in https://github.com/apache/rocketmq/pull/9511
• [ISSUE [#9501]] Correcting mismatched comments by @yx9o in https://github.com/apache/rocketmq/pull/9502
• [ISSUE [#9515]] Return origin handle to consumer when changeInvisibleDuration encounters flow limit by @qianye1001 in https://github.com/apache/rocketmq/pull/9516
• [ISSUE [#9254]] Fix combineCQ extra search commitLog files for recovery by @qianye1001 in https://github.com/apache/rocketmq/pull/9526
• [ISSUE [#9309]] Avoid the generation of dirty data in #assignResetOffset by @3424672656 in https://github.com/apache/rocketmq/pull/9310
• [ISSUE [#9530]] Reduce unnecessary memory allocation in adaptiveBackOffSpinLock by @3424672656 in https://github.com/apache/rocketmq/pull/9530
• [ISSUE [#9520]] Remove unnecessary topicConfig data version change when enable split registration by @RongtongJin in https://github.com/apache/rocketmq/pull/9521
• [ISSUE [#9254]] Optimize the logs of the message store by @qianye1001 in https://github.com/apache/rocketmq/pull/9528
• [ISSUE [#9537]] MQClientAPIFactory Implement NameServerUpdateCallback interface by @qianye1001 in https://github.com/apache/rocketmq/pull/9538
• [ISSUE [#9544]] Must correct file size when init the file segment by @lizhimins in https://github.com/apache/rocketmq/pull/9545
• [ISSUE [#9539]] Fix compare policy entry logic in ACL 2.0 by @1782935682 in https://github.com/apache/rocketmq/pull/9540
• [ISSUE [#9534]] Remove the meaningless brokerVersion by @RongtongJin in https://github.com/apache/rocketmq/pull/9533
• [ISSUE [#9555]] Fix the npe in the hashCode method when subscriptionDataSet is null by @1782935682 in https://github.com/apache/rocketmq/pull/9556
• [ISSUE [#9553]] Improve performance by avoiding repeated get(key) by @yx9o in https://github.com/apache/rocketmq/pull/9554
• Limit group max length to 120. by @lollipopjin in https://github.com/apache/rocketmq/pull/9563
• [ISSUE [#9254]] Refactor notifyMessageArriveInBatch in RocksDBConsumeQueueStore to adapt to CombineConsumeQueueStore by @qianye1001 in https://github.com/apache/rocketmq/pull/9566
• [ISSUE [#9572]] Unnecessary code comment cleanup by @yx9o in https://github.com/apache/rocketmq/pull/9573
• Fix typos by @co63oc in https://github.com/apache/rocketmq/pull/9560
• [ISSUE [#9254]] Optimize innerConsumeQueueStoreList order in CombineConsumeQueueStore by @qianye1001 in https://github.com/apache/rocketmq/pull/9599
• [ISSUE [#9596]] Optimize log in invokeSync when addr is null by @1782935682 in https://github.com/apache/rocketmq/pull/9597
• [ISSUE [#9605]] Fixed a potential resource leak by @yx9o in https://github.com/apache/rocketmq/pull/9606
• [ISSUE [#9589]] Optimize broker metrics initialization by @qianye1001 in https://github.com/apache/rocketmq/pull/9598
• [ISSUE [#9611]] Should exec callback in the Pop based on rocksdb impl by @lizhimins in https://github.com/apache/rocketmq/pull/9612
• [ISSUE [#9600]] Remove wrong logic in the callback for sending messages in rpc by @3424672656 in https://github.com/apache/rocketmq/pull/9608
• [ISSUE [#9626]] Prevent premature offset commit before consumer record flush by @lizhimins in https://github.com/apache/rocketmq/pull/9627
• [ISSUE [#9542], [#9609]] Fix bazel CI and reduce dependencies by @xuhongxu96 in https://github.com/apache/rocketmq/pull/9610
• [ISSUE [#9652]] Enhance test for MQClientInstance by @GabrielDUX in https://github.com/apache/rocketmq/pull/9653
• [ISSUE [#9648]] Fix getOffsetInQueueByTime missing boundaryType in tieredMessageStore by @GabrielDUX in https://github.com/apache/rocketmq/pull/9649
• Fix start() and shutdown() of DefaultMessagingProcessor by @redlsz in https://github.com/apache/rocketmq/pull/7838
• [ISSUE [#9663]] Adopt Maven's revision mechanism (requires Maven >= 3.5.0) by @cvictory in https://github.com/apache/rocketmq/pull/9665
• [ISSUE [#9650]] Unified FAQ related URLs by @yx9o in https://github.com/apache/rocketmq/pull/9651
• [ISSUE [#9620]] Placeholder replacement string concatenation by @yx9o in https://github.com/apache/rocketmq/pull/9621
• [ISSUE [#9581]] Optimize the resource bloat of pollingMap and topicCidMap in LMQ scenarios by @RongtongJin in https://github.com/apache/rocketmq/pull/9579
• [ISSUE [#9764]] Fix bazel-compile (ubuntu-latest) error by @yx9o in https://github.com/apache/rocketmq/pull/9675
• [ISSUE [#9666]] Add accelerated startup recovery feature for RocksDB store with SYNC_FLUSH by @RongtongJin in https://github.com/apache/rocketmq/pull/9667
• [ISSUE [#9634]] Improve BrokerContainer extensibility and module structure by @RongtongJin in https://github.com/apache/rocketmq/pull/9635
• [ISSUE [#9547]] Fix sync group if attributes isn't null by @1782935682 in https://github.com/apache/rocketmq/pull/9679
• [ISSUE [#9682]] Increase initial delay for scanning inactive brokers to 5000 milliseconds by @mxsm in https://github.com/apache/rocketmq/pull/9683
• [ISSUE [#9680]] Improve RocksDB compaction filter factory resource management by @RongtongJin in https://github.com/apache/rocketmq/pull/9681
• [ISSUE [#9677]] Resolve metrics static variable conflicts in BrokerContainer mode by @RongtongJin in https://github.com/apache/rocketmq/pull/9678
• [ISSUE [#9695]] Not use pull offset when use pop orderly consume by @lizhimins in https://github.com/apache/rocketmq/pull/9696
• [ISSUE [#9693]] Add writeWithoutMmap configuration to prevent JVM crash when device becomes read-only by @guyinyou in https://github.com/apache/rocketmq/pull/9694
• [ISSUE [#9701]] Synchronize metrics shutdown to prevent JVM crashes during broker shutdown by @guyinyou in https://github.com/apache/rocketmq/pull/9702
• [ISSUE [#9703]] Remove pull offset and reset offset when deleting topic by @qianye1001 in https://github.com/apache/rocketmq/pull/9704
• [ISSUE [#9707]] Integrate RunningFlags with MappedFile system for better error handling and state management by @guyinyou in https://github.com/apache/rocketmq/pull/9708
• [ISSUE [#9713]] Improve data consistency in DefaultMappedFile flush method by @guyinyou in https://github.com/apache/rocketmq/pull/9714
• [ISSUE [#9699]] Optimize shutdown process and resource management by @RongtongJin in https://github.com/apache/rocketmq/pull/9700
• [ISSUE [#9705]] Improve persist() method reliability to prevent broker startup failure after power outages by @guyinyou in https://github.com/apache/rocketmq/pull/9706
• [ISSUE [#9709]] Add enableRunningFlagsInFlush configuration switch for CommitLog by @guyinyou in https://github.com/apache/rocketmq/pull/9710
• [ISSUE [#9716]] Refactor: replace RandomAccessFile with FileChannel by @RongtongJin in https://github.com/apache/rocketmq/pull/9715
• [ISSUE [#9717]] Fix RaftBrokerHeartBeatManager#scanNotActiveBroker was not actually executed by @redlsz in https://github.com/apache/rocketmq/pull/9718
• [ISSUE [#9721]] Fix TimerDequeueGetService thread not exiting after shutdown by @RongtongJin in https://github.com/apache/rocketmq/pull/9722
• [ISSUE [#9689]] Fix the issue that master transfer epoch was not updated in time by @redlsz in https://github.com/apache/rocketmq/pull/9690
• [ISSUE [#9719]] Fix Static Variable Conflicts in Multi-Broker Metrics Management by @RongtongJin in https://github.com/apache/rocketmq/pull/9720
• Fix/update metadata bug in v2 config by @cvictory in https://github.com/apache/rocketmq/pull/9724
• [ISSUE [#9726]] Abstract StoreMetricsManager interface to support non-DefaultMessageStore implementations by @RongtongJin in https://github.com/apache/rocketmq/pull/9727
• [ISSUE [#9687]] Cleanup BrokerHeartbeatManager code by @majialoong in https://github.com/apache/rocketmq/pull/9688
• [ISSUE [#9732]] Fix request may still be invoked when local process timeout by @qianye1001 in https://github.com/apache/rocketmq/pull/9733
• [ISSUE [#9658]] Replace JUnit 5 imports with JUnit 4 by @yx9o in https://github.com/apache/rocketmq/pull/9659
• [ISSUE [#9743]] Fix ReceiptHandleProcessorTest by @yx9o in https://github.com/apache/rocketmq/pull/9744
• Fix typos and linguistic errors in documentation / hacktoberfest by @survivant in https://github.com/apache/rocketmq/pull/9748
• fix get dledger mod storepath by @LittleBoy18 in https://github.com/apache/rocketmq/pull/9746
• [ISSUE [#9741]] Optimize authentication whitelist lookup efficiency by @yx9o in https://github.com/apache/rocketmq/pull/9742
• [ISSUE [#9739]] Revert skipped test cases in macos environment by @lizhimins in https://github.com/apache/rocketmq/pull/9749
• [ISSUE [#9750]] Fix NPE when isAsyncSendEnable is false by @RongtongJin in https://github.com/apache/rocketmq/pull/9751
• [ISSUE [#9756]] Fix the issue where the port is always 10911 when starting broker with brokerStartup by @RongtongJin in https://github.com/apache/rocketmq/pull/9755
• [ISSUE [#9752]] Fix time dequeue latency unit to milliseconds by @dingshuangxi888 in https://github.com/apache/rocketmq/pull/9753
• [ISSUE [#9761]] Fix get authentication and authorization metadata provider error in AuthorizationMetadataManagerImpl by @majialoong in https://github.com/apache/rocketmq/pull/9762
• [ISSUE [#9758]] Fix resource filter does not take effect in the listAcl by @majialoong in https://github.com/apache/rocketmq/pull/9759
• [ISSUE [#9765]] Improve null handling in file operations by @yx9o in https://github.com/apache/rocketmq/pull/9766
• [ISSUE [#9767]] Fix incorrect decode method for UnlockBatchRequestBody in DefaultAuthorizationContextBuilder by @majialoong in https://github.com/apache/rocketmq/pull/9768
• [ISSUE [#9769]] Add tls.ciphers and tls.protocols in system properties by @carlvine500 in https://github.com/apache/rocketmq/pull/9770
• [ISSUE [#9771]] Remove static from transactionalMessageCheckListener executorService to support multi-broker in brokercontainer by @RongtongJin in https://github.com/apache/rocketmq/pull/9772
• [ISSUE [#9763]] Fix invalid user disable status check in authorization by @majialoong in https://github.com/apache/rocketmq/pull/9764
• [ISSUE [#9773]] Implement Shared RocksDB Instance for Broker Configs by @qianye1001 in https://github.com/apache/rocketmq/pull/9774
• [ISSUE [#9776]] Make SharedByteBuffer size configurable via MessageStoreConfig.maxMessageSize by @RongtongJin in https://github.com/apache/rocketmq/pull/9775
• [ISSUE [#9773]] Fix core dump when shutdown broker twice by @qianye1001 in https://github.com/apache/rocketmq/pull/9783
• Correct method name from updateDateVersion to updateDataVersion by @majialoong in https://github.com/apache/rocketmq/pull/9788
• [ISSUE [#9784]] Fix the policy comparator to prioritize DENY over ALLOW by @majialoong in https://github.com/apache/rocketmq/pull/9785
• [ISSUE [#9798]] Reduce unnecessary thread switching in sending message via proxy by @qianye1001 in https://github.com/apache/rocketmq/pull/9799
• [ISSUE [#9811]] Fixed the issue of excessive loop iterations in reput service by @lizhimins in https://github.com/apache/rocketmq/pull/9812
• [ISSUE [#9816]] Fix concurrent modify opentelemetry record in calculate consumer lag by @lizhimins in https://github.com/apache/rocketmq/pull/9817
• [ISSUE [#9813]] Fix buffer fetch thread pool starvation in tiered storage by @lizhimins in https://github.com/apache/rocketmq/pull/9818
• Fixed error message of CommandUtil.fetchMasterAddrByBrokerName by @TheRealHaui in https://github.com/apache/rocketmq/pull/9825
• [ISSUE [#9802]] Fix the issue of delete logic in tiered storage index service by @lizhimins in https://github.com/apache/rocketmq/pull/9803
• [ISSUE [#9819]] Upgrade OpenTelemetry to version 1.44.1 by @RongtongJin in https://github.com/apache/rocketmq/pull/9820
• [ISSUE [#9735]] Add TimerWheel snapshot mechanism for reliable recovery by @guyinyou in https://github.com/apache/rocketmq/pull/9736
• Add query message offset by timestamp to abstract class by @lizhimins in https://github.com/apache/rocketmq/pull/9836
• [ISSUE [#9821]] Fix getBrokerConfig no response issue for empty parameters by @yx9o in https://github.com/apache/rocketmq/pull/9822
• [ISSUE [#9794]] Change JDK prerequisite from 1.7+ to 1.8+ in documentation by @yx9o in https://github.com/apache/rocketmq/pull/9795
• fix controller mode scanControllerAddress by @LittleBoy18 in https://github.com/apache/rocketmq/pull/9845
• [#ISSUE 9841] Improve Resource Management in TimerWheel to Prevent File Handle Leaks by @guyinyou in https://github.com/apache/rocketmq/pull/9842
• [ISSUE [#9632]] Fix: Pop Long-polling Not Awakened for V1 Retry Messages by @qianye1001 in https://github.com/apache/rocketmq/pull/9828
• [ISSUE [#9838]] IndexStoreService use forceShutdown when disk is not writable by @RongtongJin in https://github.com/apache/rocketmq/pull/9839
• [ISSUE [#9834]] Support writeWithoutMmap in IndexStoreFile by @RongtongJin in https://github.com/apache/rocketmq/pull/9835
• Fix resource leak in IndexStoreFile shutdown by @RongtongJin in https://github.com/apache/rocketmq/pull/9840
• [ISSUE [#9593]] Optimize Deflater resource release by @yx9o in https://github.com/apache/rocketmq/pull/9594
• [ISSUE [#9847]] Reduce lock contention on the HandleData object to prevent threads from hanging by @qianye1001 in https://github.com/apache/rocketmq/pull/9848
• [ISSUE [#9857]] Optimize the performance of the notification by removing unnecessary retry checks by @qianye1001 in https://github.com/apache/rocketmq/pull/9858
• [ISSUE [#9807]] Optimize log.warn Time Consumption in the send Method by @yx9o in https://github.com/apache/rocketmq/pull/9808
• [ISSUE [#9737]] Fix client memory leak on connection failure by @qianye1001 in https://github.com/apache/rocketmq/pull/9740
• [ISSUE [#9849]] Correct ACL cache loader error log in ClusterMetadataService by @majialoong in https://github.com/apache/rocketmq/pull/9850
• [ISSUE [#9741]] Optimize authorization whitelist lookup efficiency by @majialoong in https://github.com/apache/rocketmq/pull/9760
• [ISSUE [#9809]] Fix NPE in getAcl when subject is null by @majialoong in https://github.com/apache/rocketmq/pull/9810
• [ISSUE [#9789]] LitePullConsumer supports manually adding subscription reported in Heartbeat by @qianye1001 in https://github.com/apache/rocketmq/pull/9790
• [ISSUE [#9862]] [Bug] Fix the issue of missing bornTime in POP requests when broker ACL is enabled by @dingshuangxi888 in https://github.com/apache/rocketmq/pull/9863
• [ISSUE [#9870]] Ensure metadata provider cache executors are shutdown correctly by @majialoong in https://github.com/apache/rocketmq/pull/9871
• [ISSUE [#9779]] Fix the promblem of 'DirectBuffer will cause error at 9+ JDK version'. by @sinberCS in https://github.com/apache/rocketmq/pull/9801
• [ISSUE [#9663]] Add flatten-maven-plugin plugin by @lizhimins in https://github.com/apache/rocketmq/pull/9866
• [ISSUE [#9816]] Fix calculate consumer lag with opentelemetry by @lizhimins in https://github.com/apache/rocketmq/pull/9873
• [ISSUE [#9875]] Optimize the RocksDB config shutdown logic when useSingleRocksDBForAllConfigs is set to true to prevent JVM crashes. by @RongtongJin in https://github.com/apache/rocketmq/pull/9874
• [ISSUE [#9734]] Prepare to release Apache RocketMQ 5.3.4 by @lizhimins in https://github.com/apache/rocketmq/pull/9861

New Contributors

• @hepengju made their first contribution in https://github.com/apache/rocketmq/pull/9408
• @EnableAsync made their first contribution in https://github.com/apache/rocketmq/pull/9483
• @huanzhikang made their first contribution in https://github.com/apache/rocketmq/pull/9492
• @woaishixiaoxiao made their first contribution in https://github.com/apache/rocketmq/pull/9498
• @Allenhuangrui made their first contribution in https://github.com/apache/rocketmq/pull/9480
• @AROP-REPLICATION made their first contribution in https://github.com/apache/rocketmq/pull/9459
• @datagramsocket made their first contribution in https://github.com/apache/rocketmq/pull/9511
• @1782935682 made their first contribution in https://github.com/apache/rocketmq/pull/9540
• @co63oc made their first contribution in https://github.com/apache/rocketmq/pull/9560
• @xuhongxu96 made their first contribution in https://github.com/apache/rocketmq/pull/9610
• @GabrielDUX made their first contribution in https://github.com/apache/rocketmq/pull/9653
• @cvictory made their first contribution in https://github.com/apache/rocketmq/pull/9665
• @majialoong made their first contribution in https://github.com/apache/rocketmq/pull/9688
• @survivant made their first contribution in https://github.com/apache/rocketmq/pull/9748
• @TheRealHaui made their first contribution in https://github.com/apache/rocketmq/pull/9825
• @sinberCS made their first contribution in https://github.com/apache/rocketmq/pull/9801

Full Changelog: https://github.com/apache/rocketmq/compare/rocketmq-all-5.3.3...rocketmq-all-5.3.4