Changelog

v0.25.0-rc1

June 22, 2026

• show running_apps in instances table · #630
• expose running app count fetch failures and reduce fetch concurrency · #636
• release timeout ticket budget · #635
• correct batch kill progress and failure counting · #629
• bump version to 0.25.0 · #637
• disable flaky test of graceful_shutdown_test · #632
• fix typo of kerberos · #631

v0.24.0-rc1

June 1, 2026

• add support of server tags modification via riffle-ctl · #626
• implement O_DIRECT append aligned with sync_io · #623
• streaming parser EOF hang on mid-frame close · #624
• bump version to 0.24.0 · #628

v0.23.0-rc1

May 25, 2026

• add metrics for connection buffer capacity tracking · #618
• add streaming urpc parsing for SendShuffleData · #617
• bump version 0.23 · #619

v0.22.0-rc1

April 20, 2026

• only activate hard_split when client partition reassign is enabled · #615
• increase max_connections to 200K · #614
• add await-tree and more logs for different phases of one request · #612
• ensure await-tree is registered only once to prevent significant performance regression · #609
• Subtract metric of read_ahead_active_tasks in ReadAheadLayerWrapper · #606
• bump version to 0.22.0 · #611
• fix(localfile)(uring): writing iovec chunked on exceeding UIO_MAXIOV limit · #608

v0.21.0-rc1

March 24, 2026

• Add histogram for RPC batch data bytes in SendDataRequestCommand · 49423cc
• allow client to configure app heartbeat timeout · #599
• add metric of urpc_request_number · #596
• Add metric of time spent (ms) looking up memory spill partitions · #595
• Log the partitions that are split/huge for the first time · #594
• Abort core-affinity in aarch64 to improve performance · #578
• Dedicated buffer size tracking to speed up spill lookup · #576
• Support read all for sync_io at IoMode::DIRECT_IO · #582
• Align with the latest profiling in README · #577
• Try to add agents.md · #571
• Adjust uring default threads to 2 · #570
• avoid incorrect purge by rechecking app heartbeat timeout · #598
• Distinguish ENOSPC from disk corruption in health check · #589
• fix use-after-move bug for splice_pipe · #579
• Potential file leak on async-task missing · #572
• Bump version to 0.21.0 · #603
• feat(1/N)(coordinator): Introduce coordinator module · #574

v0.20.0-rc2

February 4, 2026

• Distinguish ENOSPC from disk corruption in health check (#589) · #591

v0.20.0-rc1

December 31, 2025

• Add metric for splice · #568
• Standardize naming by hard-split and add metric · #565
• Introduce a pluggable mem-buffer mechanism and an optimized staging buffer impl · #564
• Fallback to psync engine when sendfile is selected · #562
• Refactor and introduce the urpcReadIoMode for client · #560
• Introduce zero-copy reads to socket using splice · #559
• Only make this valid on Linux · #558
• Respect io-uring config options · #551
• Activate anolisos8 + arm64 release · #550
• Add support of io_uring engine (experimental) · #549
• Use the HARD_SPLIT_FROM_SERVER code to followup uniffle impl · #547
• Upgrade integration env to anolisos8 · #544
• Quick start to build the riffle cluster · #535
• Must register hard-split metric · #569
• Data inconsistency due to hanging pointer of written bytes · #553
• Hang with sql-tests · #539
• With token to avoid api limit on setup-protoc · #538
• flaky test of urpc#test_client · #537
• Bump version to 0.20.0 · #567
• Add uring quick-start · #561
• Remove v2 suffix for memory buffer · #548
• Remove unrelated tests · #533
• feat(ci)(3/N): Add prometheus + grafana in integration test · #540

v0.19.0-rc1

December 3, 2025

• Trigger partition split when storage is unhealthy · #532
• Use spark.rss.riffle.getMemoryDataUrpcVersion for client to control compatible urpc version · #529
• Introduce isEnd to indicate EOF on getting memory data · #527
• Bump version to 0.19.0 · #536
• Upgrade version to 0.18.1 · #528
• feat(ci)(2/N): Add tpcds test with uniffle + spark · #534
• feat(ci)(1/N): Introduce integration tests with uniffle + spark · #531

v0.18.2

January 21, 2026

• Abort core-affinity in aarch64 to improve performance · #578

v0.18.1

December 29, 2025

• Activate anolisos8 + arm64 release · ed0fa55
• Bump version to 0.18.1 · 0f01fa5

v0.18.0

November 19, 2025

This stable release promotes v0.18.0-rc2 without additional source changes.

v0.18.0-rc2

November 19, 2025

• activate amd64 + anolisos release · #526

v0.18.0-rc1

November 14, 2025

• Add support of partition_stats management · #521
• Always show the app’s hdfs root · #516
• Potential duplicate partition_stats report bound by the same task_attempt_id · #524
• Implement display for PurgeReason to show more infos · #522
• Bump version to 0.18.0 · #525
• Delete unused proto fields for ReportShuffleResultRequest · 15bfe0b

v0.17.0-rc1

October 15, 2025

• Make jemalloc as default feature · #515
• Replace all dashmap to ddashmap with fast hash · #514
• Refactor ddashmap to set up the hash algo · #512
• Add support of kill with blocking wait · #510
• Refactor admin http api to add more api · #509
• Reduce binary size · #507
• Correct the read ahead task metrics on deletion · #513
• Remove unix specified import to avoid compile failure in windows · #511

v0.16.0

September 16, 2025

• Only log active urpc request in await-tree · #506
• Always use Bytesize crate to remove readable_size · #505
• Add default runtime config test · #504
• Bump version to 0.17.0 · #503

v0.16.0-rc2

September 16, 2025

• Disable fail fast and anolisos8 on release workflow · #502

v0.16.0-rc1

September 15, 2025

• Add root into the read-ahead processor await-tree log · #501
• Normalize app_id output · #500
• Purge apps when accepting force-kill signal · #499
• Distinguish metrics based on read-ahead being disabled · #495
• Add metric to observe small range read latency/size · #497
• Accept read plan and then to prefetch to adapt AQE skew optimization · #488
• followup #481 to respect options setting by client for read-ahead batch number/size · #485
• Correct hit metric and introduce 2 metrics to indicate duration · #484
• Introduce seperated options to control ahead batch number/size for client · #481
• Mark partial logs with long-running tag · #479
• Await-tree log with read/writer options · #477
• Missing task_id for read plan · #491
• Noisy log when deleting in read ahead io layer · #472
• Incorrect log when conf key updating · #468
• Hang on holding load_task · #476
• Make sequential param as false by default · #474
• Bump version to 0.16.0 · #473
• feat(read-ahead)(read-plan): Use dedicated thread pool for read-plan processor · #494
• feat(read-ahead)(read-plan): Add hit/miss metrics · #493

v0.15.1

September 11, 2025

• Add metric to observe small range read latency/size · #497

v0.15.0

August 28, 2025

This stable release promotes v0.15.0-rc3 without additional source changes.

v0.15.0-rc3

August 28, 2025

• Hang on holding load_task · #476

v0.15.0-rc2

August 27, 2025

• Make sequential param as false by default · #474

v0.15.0-rc1

August 25, 2025

• Introduce optional huge_partition_spill var to disable flushing to HDFS · #467
• Simplify pre-check logic for hdfs · #455
• Invalid upload operation for centos7 binaries · #462
• Always print normalized app_id in logs · #460
• Use mut self for builder-style chaining for ReadingViewContext · #459
• Bump version to 0.15.0 · #457
• fix(read-ahead)(5/N): Only record data file reading latency · #463
• feat(read-ahead)(4/N): Add more metrics · #461
• feat(read-ahead)(3/N): Complete read-ahead configuration options · #458
• feat(read-ahead)(2/N): Infer localfile sequential read to activate read ahead · #456

v0.14.1

August 14, 2025

This stable release promotes v0.14.1-rc1 without additional source changes.

v0.14.1-rc1

August 14, 2025

• Remove metric label of huge_partition_number · #444
• Too frequent log output on urpc send_data · #452
• Always print normalized app_id in logs · #442
• README update again · #450
• README update · #445

v0.14.0

August 13, 2025

This stable release promotes v0.14.0-rc5 without additional source changes.

v0.14.0-rc5

August 13, 2025

• Correct artifacts name · #449

v0.14.0-rc4

August 12, 2025

• Multi artifacts for one arch and os · #447

v0.14.0-rc3

August 12, 2025

• Unknown flag when uploading to release page · #446

v0.14.0-rc2

August 12, 2025

• Use fixed rustc version to bypass build failure on macos · #443

v0.14.0-rc1

August 8, 2025

• Add rpc duration to inspect · #441
• Add server status transition reason · #440
• Enable keep_alive and nodelay for connection · #437
• Add support of task_ids filter on memory getting rpc · #433
• Add cluster ids config to simplify specify coordinator url · #430
• Remove unused param and make updated status as enum · #429
• Fallback to localfile on huge partition writing hdfs failure · #415
• Detailed log on flushing failure · #416
• Add license file · #417
• Add default tags · #413
• Add support of read ahead · #411
• Upgrade hdrs version to record opening file error · #408
• Add fallback mechanism to avoid panic on application id parsing · #407
• Align with the same prefix of spark.rss.riffle for all client options · 1003155
• Add hdfs append action to test · #400
• Add precheck mechanism to avoid silent failure · #405
• Refactor the let chains to fix release CI · #422
• Remove HDFS main app dir when explicit-unresgiter or heartbeat-timeout · #420
• Missing component identifier in asset file name · #402
• Add updated cargo.lock of upgraded hdrs version · #409

v0.13.0

July 24, 2025

This stable release promotes v0.13.0-rc2 without additional source changes.

v0.13.0-rc2

July 24, 2025

• Upgrade hdrs version to record opening file error · #408

v0.13.0-rc1

July 22, 2025

• Only one thread can create dir on hdfs and upgrade hdrs version to print stacktrace · #396
• Support per-application toggle for sendfile feature · #393
• Respect client-side RSS configs when applying app-level options · #391
• Use sendfile to optimize file-to-socket transfers on Linux · #379
• Activate log append into stdout for default log · #384
• Replaced BufWriter with an explicit write buffer for TcpStream · #378
• Align with log options · #375
• Build artifacts for labeled PR · #376
• Avoid panic on hdfs client creating failure · #398
• Reserve log guard to make it valid · #394
• Simplify sendfile logic and fix sendfile hang · #389
• logforth compatibility with version 0.21.6 · #373
• LocalIO trait · #377
• Remove noisy log about ticket checking · #382
• Upgrade logforth to 0.26.2 to fix rolling · #380
• Upgrade logforth version to 0.26.1 · #372

v0.12.0

June 19, 2025

This stable release promotes v0.12.0-rc4 without additional source changes.

v0.12.0-rc4

June 19, 2025

• Only retrieve necessary tables for query · #359
• Incorrect logic of is_huge_partition · #360
• Remove reject mod · #356

v0.12.0-rc3

June 17, 2025

• Make service unhealthy if IO operation timeout · #354
• Correct apps purge due to incorrect timeout · #355
• Potential invalid huge partition mark · #353

v0.12.0-rc2

June 16, 2025

• Compile failure with feature hdrs · #352

v0.12.0-rc1

June 16, 2025

• Add urpc decoding bench · #350
• Remove app level statistics with metrics format · #347
• Add riffle-ctl introduction · #348
• Expose partition split triggered info into http api · #346
• Add support of kill command · #338
• Add the QPS metric · #342
• Refine index file format capacity · #340
• Reduce too many await-tree and refine hashmap capacity on parsing · #339
• Introduce ApplicationId for better hash/allocation/clone performance · #337
• Skip strings and numa awareness · #336
• Use flatten encoding for send_shuffle_data / report_shuffle_result · #332
• Add bench for protobuf flat/nested structure · #330
• Use extend to add multi blockIds of treemap for performance · #329
• Directly serialize for only one partition on blockIds getting · #328
• More precise rpc batch data size bucket · #327
• Split into multiple files for app_manager · #325
• Use block pool for composed_bytes to reduce long-tail latency · #322
• Remove lock on localfile reading · #321
• Refactor extended dashmap for app manager · #320
• Detect disk write+read with more data by direct IO · #345
• Disable disk detection on test env · #351
• Bump version to 0.12.0 · #349
• Revert "feat: Use block pool for composed_bytes to reduce long-tail latency (…" · #323

v0.11.0

June 3, 2025

This stable release promotes v0.11.0-rc2 without additional source changes.

v0.11.0-rc2

June 3, 2025

• Disable parking_lot deadlock detection by default · #319

v0.11.0-rc1

May 28, 2025

• Eliminate cost of partition uid construction on partition limit/split disabled · #317
• Replace rwlock with parking_lot mutex in memory buffer access · #316
• Await-tree too much/heavy spans · #314
• Introducing dashmap extend to unify invoking sides · #313
• Unify rpc batch data bytes into one metric with different labels · #308
• Add received bytes in active app infos · #307
• Pre-check target status and echo error failure · #306
• Collect read local data bytes with histogram format · #305
• Introducing decommission_grpc_mode in update status command of ctl · #304
• Automatically reload tables on the PostgreSQL server · #302
• Using more fast hasher in app hotpoint dashmap · #301
• Collect localfile index data bytes with histogram format · #300
• Upgrade await-tree for better performance · #293
• Remove unnecessary files to free up disk space · #294
• Introducing disk-read-bench to test read performance · #291
• add PostgreSQL protocol support for better inspection · #284
• Decomposed into sub-crates to optimize build performance · #288
• Pluggable historical apps store · #282
• Optional reconfigurable conf · #281
• Using mem table to replace csv disk datasource · #275
• Eliminate outdated config option of dashmap shard · #276
• Remove unnecessary raw bytestring value to eliminate copy cost in config_ref · #273
• Tolerating partial coordinator heartbeat connection failures · #270
• Decrease grpc number metrics in some abnormal cases · #309
• Panic on cpu profiling · #290
• Invalid on_change on duplicate registered same conf key · #280
• Upgrade version to 0.11.0 · #315
• With bytes size in rpc localfile get data/index · #312
• Log throttle layer being enabled · #271

v0.10.0

May 22, 2025

This stable release promotes v0.10.0-rc3 without additional source changes.

v0.10.0-rc3

May 22, 2025

• fix: Panic on cpu profiling #290 · a2be6d4

v0.10.0-rc2

May 14, 2025

• Dedicated options to control read/write throttle · #267
• Re-introducing throttle mechanism · #263
• Completion all status updating ctl interface · #260
• Simplify release workflow · #259
• Add support of riffle-ctl macos release CI · #258
• Incorrect shuffle-server-ip due to the random port initialization sequence · #264

v0.10.0-rc1

May 8, 2025

• Dynamic concurrency limiter for event bus · #255
• Optional unhealthy status on initial starting · #256
• Introducing disk bench/profiler commands to inspect IO performance · #253
• Distinguish dynamic or static config options for performance · #248
• Throw root cause on hdfs operations failure · #245
• Inject ports into process env · #242
• Rename update state http API of admin handler · #220
• Using dynamic http port to fetch · #235
• Add support of deletion retry · #234
• Introduce option for service with random ports if used · #231
• Expose version + build date into http api · #229
• Maximize available disk space for github action · #215
• Make exceeding duration IO exit for hdfs append operation · #203
• Add more logs when decommission · #208
• Fallback to capacity/2 when acquired ticket > capacity in throttle layer · #254
• Correct hdfs writing failure log · #246
• Remove child log path with server_id · #240
• Activate internal grpc service · #226
• No space left on device when compiling · #218
• Introduce on_change for callback · #249
• Introducing multi pluggable layers for IO · #232
• Bump version to 0.10.0 · #257
• Upgrade hdfs-native to 0.11.2 for better write stability · #244
• Eliminate all compiler warnings · #241
• feat(ctl)(4/N): Support pipeline mode for decommission · #225
• fix(heartbeat)(2/N): Reset is_healthy rpc param when state is not active · #222
• feat(heartbeat)(1/N): Align with uniffle heartbeat protobuf · #219
• feat(ctl)(3/N): Add historical_apps and active_apps tables to query · #213
• feat(ctl)(2/N): Add support of node update command · #211
• feat(ctl)(1/N): Add query command for ctl · #190

v0.9.1-rc1

May 20, 2025

• Panic on cpu profiling · 68ae68a

v0.9.0

April 16, 2025

This stable release promotes v0.9.0-rc5 without additional source changes.

v0.9.0-rc5

April 16, 2025

• Activate internal grpc service · #226

v0.9.0-rc4

April 15, 2025

• Reset is_healthy rpc param when state is not active · #223
• Invalid decommission operation · 6e7b796

v0.9.0-rc3

April 15, 2025

• Invalid decommission operation · #221

v0.9.0-rc2

April 9, 2025

• Bump version to 0.9.0 · #210
• Update README · #209

v0.9.0-rc1

April 9, 2025

• Add context for local io failure · #201
• Mix different apps api into one · #199
• Add support of heap profile with multi format · #198
• Add support of cpu profile with multi format · #197
• Unify index encode/decode into one place · #155
• Expose http api to decommission · #194
• Bound server_id into log parent folder · #193
• Remove legacy memory profile dump with pprof format · #195
• Introducing option to control disk pre launch purge · #192
• Support decommission grpc api · #191
• Bound shuffle_server_id into disk detection file name to ensure unique · #189
• Involve version into the prometheus metric label · #185
• Activate partition limit whatever partition split happens · #205
• Flaky test of get_data_from_localfile · #187
• Pre-check partition-limit and partition-split threshold value · #206

v0.8.2

March 27, 2025

This stable release promotes v0.8.2-rc1 without additional source changes.

v0.8.2-rc1

March 27, 2025

• Activate conf reload for partition limit related conf · #183
• Add fallback for byteString in conf reload · #180
• Generics to simplify reconf value invoking · #179
• Add support of conf reload · #178
• Add deadlock detection · #177
• Add panic hook to catch it · #176
• Attach tokio runtime name into background await-tree logs · #175
• Align arm binary name with other binaries · #174
• Bump version to 0.8.2 · #184
• Eliminate all compiler warnings · #181

v0.8.1

March 19, 2025

• Upload binaries to corresponding release page · #172
• Rename release binary from uniffle-worker to riffle-server · #171
• Attach tag/branch info into release artifact · #170
• Bump version to 0.8.1 · #173

v0.8.0

March 19, 2025

• Release build with anolisos8.8 on amd64 · #165
• Add support of partition split · #164
• followup to ignore non-huge-partition apps · #163
• followup to activate historical app http json handler · #162
• Support historical apps recording · #161
• Token bucket IO limiter · #157
• Add support index consistency detection for debug · #153
• Add support of heap profile flamegraph · #145
• Add support of async watermark-spill trigger · #144
• Aggressively moving forward low-watermark on sensitive spill · #142
• Unify await_tree spawn runtime · #137
• Add await-tree for heartbeat task · #136
• Log watermark spill conditions to check correctness · #135
• Only mark as unhealthy when node app_running_num > 50 · #159
• Potential incorrect partition lock deletion due to the shuffle level purge · #156
• Align with the meaning of empty option of io_scheduler to disable it · #152
• Log flush error on flushing failure · #151
• Activate huge_partition limit mechanism due to incorrect partition_ids · #149
• Replace add with sub for GAUGE_MEMORY_SPILL_IN_FLIGHT_BYTES_OF_HUGE_PARTITION · #141
• Exclude in_flight_bytes when looking up candidates · #140
• Forget to register metric of memory_spill_in_flight_bytes_of_huge_partition · #139
• Bump version to 0.8.0 · #167
• Add more mem infos when huge partition limited happens · #147
• Add detailed operations on await-tree log · #143
• Bump version to 0.7.1 · #138

v0.7.0

March 4, 2025

• Scope huge_partition’s flight bytes into the watermark spill calculation to avoid backpressure · #130
• Upgrade clap to version 4.x · #127
• Unlimited dispatcher event-bus concurrency · #121
• Add grafana dashboard template · #119
• Expose http json api to get apps info · #117
• Scope independent grpc request paths into qps metrics · #115
• Introduce local_disk_service_used metric · #114
• Adapt with multi different disks number on min_available_disk_number config · #109
• Add disk explorer to detect bandwidth · #105
• More easier to specify io-scheduler bandwidth config option · #104
• Add health check policy for potential system hang · #99
• local_disk_used_ratio is always 0 · #113
• Bump the version to 0.7.0 · #133
• Log the huge partition limitation configs · #131
• Rename the data validation tools to ctl · #126
• Remove unused code · #123
• Remove the legacy jemalloc heap profile way · #118
• Delete async local io code · #110
• Remove unused structure · #102
• Use parking_lot mutex to replace std for better performance in await-tree · #122

v0.6.0

February 20, 2025

• Add the metric of resident bytes · #97
• Sync all on buffer write to avoid latter page cache flush back storm · #93
• direct_io with timeout mechanism to fast fail when io hang · #80
• Inject cpu_arch into metric system and http service · #79
• Trigger release binary on master push · #78
• Introduce option to control read/append direct io · #72
• Build binary that run on anolis aarch64 · #67
• Enable memory-prof feature when releasing · #65
• Exceed capacity permits request to fallback to the biggest capacity buffer · #95
• Enable O_DIRECT + O_NOATIME flags on direct read · #88
• CI failure of get_data_from_localfile · #90
• Respect read/append independent direct-io option · #87
• Fast fail on throwing dir_not_found to ignore events dropped metrics · #84
• Add local_disk_healthy metric · #81
• Unset runtime blocking thread num to activate parallel ability · #76
• Correct status code of huge partition requirement rejection · #66
• [followup] debug: Correct partitionId to log · #92
• Log duration when getting local data + index · #91
• [#74] fix(single-buffer-flush): Avoid using the global lock to skip some single partition flush logics · #75

v0.5.0

February 11, 2025

• Remove duplicate CI workflows to speed up and add test on aarch64 · #55
• Respect the log directory for logforth · #61
• Lazily initialize hdfs client to reduce unnecessary memory footprint · #58
• Avoid sync block_on when registering · #56
• Add symbolize for memory profile · #54
• Add the metric of memory_allocated_tickets_num · #52
• Upgrade to actions/upload-artifact@v4 · #53
• Introduce option to mark service unhealthy when exceeding the memory allocation threshold · #51
• Report block id number into metric system · #50
• Introduce pluggable block id manager · #41
• Release workflow with hdrs feature · #59
• Correct check of allocator allocation size in health service · #60
• Directly return when app has been purged · #38
• Collapse partial section · #63
• Update README · #62

v0.4.1

January 16, 2025

• Add more await-tree logs on getting block_id bitmap · #39
• Inject compile git commit id into the runtime · #37
• App clone when registering for global race condition to speed up · #36
• Avoid hang when getting blockId bitmap · #35

v0.4.0

January 15, 2025

• Try to merge multi blockIds bitmap into one by shuffleId to reduce memory footprint · #25
• Purge blockId bitmap on shuffle level deletion · #24
• Make debug level to 1 for better pprof visualization · #22
• Use take to avoid allocation when releasing ioBuffer · #21
• Auto return back to the buffer pool when buffer is useless · #20
• Support direct io for predictable io performance · #19
• Only delete hdfs files with owned shuffle-server id · #32
• Align with the internal blockId config · #33
• Correct partition number metric · #30
• Correct snake case of getHdfsDelegator · #27
• Allow dead_code and unused import or vars · #26
• Update croaring to the version 2.x · #23
• Update README · 888c959
• With offset in reading await-tree log · d297d29
• Add support of customized buffer percent · 7e44106
• Balance append + read operations · 9c152bd
• Collect read memory allocation time · 17d4bd9
• Scope io duration with timeout · 05feaac

v0.3.1

December 24, 2024

• Record health stat transition in logs · 89105af
• Extra health check for disk used ratio to fallback assp · 566fc1c
• Local calculate for p99/pxx · 74b3719
• Introduce logforth crate to rolling file and auto purge · 21d6af8

v0.3.0

December 20, 2024

• Simplify multi level event bus · 6f8e2b5
• Introduce unified metrics for rpc latency · 6e5a57b
• Mark hdfs unhealthy once the oom occurs · 20d332e
• More infos in localfile io operations · 4f445b7
• Set the concurrency limit followed by the runtime thread num by default · 4b00aa2
• Scope the io operations into the runtime dedicated spawn_blocking thread_pools · 758e9e4
• Support fallback when warm is unhealthy · 269d79f
• try(localfile)(3/N): correct the data file offset · a8aea68
• amend the lazily freeze operation · 732e5f1
• Freeze data only when flushing to reduce resident memory size · 1f1e8e1

v0.2.0

December 10, 2024

• Print more logs on fs failure operations · 480e30c
• Add all config options · 3c59aed
• Add support of customize labels · d67201b
• Only unregister huge partition metric if exists · e3c8190
• Add partition number into app dashboard · fe4d2c5
• Add basic apps dashboard · aad5445
• Add app level flushed bytes metrics · 257c1c9
• Correct illegal filter out condition when topN data size <=0 · da77fb7
• Correct date + partition number in apps dashboard · f503133
• Correct duration in app dashboard · 7be3c67
• Make the invalid_block_id as -1 rather than 0 to fix block_id=0 reading dead loop · a4f8e44
• Remove the crc verfication to reduce cpu cost · d764bd4
• correct html title · 40e7b63
• Filter out <=0 resident apps · 0de052d

v0.1.0

December 5, 2024

• Add basic release workflow · 3ef8b5e
• Add disk used ratio metric · c651ed5
• Add gauge metric of huge partition number · c680dbb
• Add long tail latency of transport log to analyze · a10b7a9
• Introduce storage level flushing size histogram · 071c9cd
• Fine-grained event metrics for localfile/hdfs flushing · 92054a9
• Introduce option to control the huge partition flushing to hdfs threshold to avoid too much small events · 7e0dfca
• Add huge partition number metric · 04ac7ce
• Output huge partition marking · beb93f6
• Catch up the localfile/hdfs underlying flush failure · b443774
• Add support of partition number metrics · cb3c718
• Provide hdrs as another hdfs writing choice · 9e10a66
• Make hdfs dependencies as the pluggable choice · 6fcd537
• Introduce heartbeat interval option of 2 seconds by default · 12a1ac5
• Support jemalloc pprof to simplify memory profile · ef76731
• Introduce service_is_healthy metric · 48a3c65
• Introduce independent health service for more policies · 38f205c
• Apply the unified rejection policy gateway · a697611
• Add r1_tool to check the data consistency · f5b1120
• Support kerberos kinit and update · 391f074
• Add support of writing multi same partition files at the same time · 99ff722
• Reduce io operations · f39befb
• Add metrics about index file reading · be404be
• Use the independent read runtime to delegate reading to avoid effecting grpc runtime · 921a188
• Add metrics/logs about disk reading · a620770
• Add support of specifying concurrency · 8b2b492
• Enable the hierarchy event bus · 4c51c18
• Support hierarchy event bus for spill · 15bfa50
• Add test case about app is not found · bb924d4
• Check app valid when spill or get · c79b195
• Add test case about spill fail and then drop · 1e3d08a
• Introduce the unified rejection policy framework · e64723d
• Create spill test arch · 963b336
• Add support of single buffer spill · e2d2fba
• Amend to isolate localfile + hdfs writing runtime · 7077bed
• Use the once cell to replace the ugly dashmap container only for later initialization in event bus · 007e1f7
• Implement huge partition -> hdfs directly · e8e575b
• Isolate storage and app_manager initialization · 097f7b0
• More quick to watermark spill · dd8812a
• Distinguish in queue/in flushing of memory spill · 039553a
• Bind the every grpc thread into fixed physical core to improve performance · 65c2e74
• Add await-tree monitor when executing by subscribers · 5bbaea5
• Introduce bps metrics for disk · 6458b8f
• Introduce io counter metrics for disk · fb11be0
• Introduce disk_write_buf_capacity to control the buf size · 98209aa
• Introduce event bus · 35342a7
• Clear up all previous disk files when initializing localfile store · b329491
• Use the BufWriter to reduce system call invoke for performance · 3417145
• Add metrics of disk append/read/stats operations time histogram · 4638368
• Collect spill batch size histogram metrics · d791807
• Collect histogram metric for bytes freeze when getting memory data · db101f5
• Use dedicated urpc runtime pool for urpc · 7b16dea
• Eliminate unnecessary bytes copy when flushing · d6fb242
• add more await-tree for disk operations · 94ac95c
• Add await-tree support for appManager · cdedda6
• Print out await-tree tasks with sorted registry ID · dfb7b42
• Set the default feature with hdfs,jemalloc,allocator-analysis · 178bfd6
• Add more operations await for disk await-tree · 0b7c834
• Add await-tree for ticket scheduled check operation · 39d8f50
• Add support of parking_lot deadlock detection · fd109e7
• Make local disk write all async · 834bb9f
• Avoid re-calculate composed bytes total len · 4aceaef
• Add more logs about appIds for those timeout tickets owner · 1a91edc
• Sort the blockId by task attempt id for AQE · 9b821b6
• Introduce the mimalloc · 4c17f42
• Support alternative memory allocated analysis · 31d8e60
• Fast fail when port has been used · c7f5fc0
• Output more logs about heartbeat timeout · e1a042a
• Remove the purged appId topN metrics · 4e55ccf
• Unify setting the default value for some config options · dcd802e
• Enable huge partition limitation · 31d5b97
• Refresh heartbeat time for every rpc especially for spark driver hang · 6a4289b
• Support spill to HDFS · 626b9a8
• Avoid extra mem allocation when reading · da03b40
• Introduce the composed byte for multi bytes · b0c4e01
• Avoid extra mem allocation to speed up reading · beab85a
• support await-tree tracing for request · 10081c5
• add some logs when encountering unexpected cases · 71eb1b1
• dump disk checking by await tree · ab3c6bf
• add metric of connection number · eaadf2c
• setup metrics · 484802c
• implement get_local_data rpc · 79bbe0b
• implement get_memory_data · 6e796b8
• improve performance to make socket shared by multiple listener · 1773a1e
• segment lock concurrency from 4 -> 10 · 56aff08
• support initial segment lock · a8724b3
• expand the grpc paralleism · 651ede6
• Introduce the memory buffer to speed up · 6412127
• assp release buffer lock · 2d61006
• make the spill event triggered by the dedicated runtime · 465506a
• Add disk capacity metric · c85b9fc
• Align with the uniffle community protoc · ec995ea
• Register total_huge_partition_number · 44bb3a8
• Only exec delete hdfs dir when having data · 494b7e6
• Always getting staging size to check whether exceeding single buffer spill threshold · 8aa4ff9
• Correct metrics registry · 4287afd
• test codebase failure · 4c59e86
• Support delete dirs recursively · 17ef839
• Correct append operation · 703df58
• Avoid holding too much time of dashmap get_mut · 0907b50
• Touch the file when creating another retry attempt append file · a042f7e
• Append another retry file when appending complete failed · 3643296
• Incorrect semaphore index retrieving · 4c64a88
• Correct parent dir creating when inserting partition data · 2032754
• CI failure · e444f0e
• spill failure when memory_localfile storage type is enabled · 802bd77
• Correct the transport time metrics of grpc · c9bcfd5
• Make in_flight bytes size bound into the spill event lifecycle · 5e7d890
• Avoid panic when file cache is missing · 69d607e
• Zero data size when initializing writing ctx · 7cd9c59
• Avoid multi initializing storage · b277fdf
• Fix embedded worker startup · 382d404
• Critical bug of appending with one batch write all to speed up IO performance · 7248d56
• Disable default features · 417c06d
• Correct app_number metric · 952a220
• Correct the spill failure metrics and drop data directly when disk is unhealthy · 9457dce
• Return the concrete status code when rejection by huge partition limit · 0a05d20
• Using the push_add when pushing prometheus · e95ea38
• Dec memory_spill_to_localfile/hdfs although spill failure · 1ff42db
• Only dec app_number metrics when purging the whole app data · 7dcedd2
• Avoid eating the unexpected error when spill · 0b0267d
• Try to fix overflow of total_localfile_used · 5378b85
• Release the memory data and drop events when disk is unhealthy · a335389
• Push metrics with worker id rather than ip · 1196742
• Throw detailed error when registering failure · 398e488
• Correct the log config and output all the config after log is initialized · 8378d30
• Re-enable the CI · b6abaeb
• Correct blockIds when the multiple partition blockids request · 9956fb2
• fill the help placeholder · 9e5fede
• some features 1. using the runtime manager to spawn background task 2. introduce the evict timeout ticket number · 952b2a6
• fix all failure tests · 577107d
• amend implement get localfile data rpc · 4ee556f
• avoid explicitly type conversion when getting urpc · d6ebb38
• skip body len · a48d6f4
• ignore empty 0 · 33cd958
• skip checked bytes in check frame · 38f087a
• correct the spill size calculation · a5f7071
• increase the max concurrency limit from 40 -> 2000 by default · 82438cb
• correct the transport time for grpc · 14d0a11
• Refactor http monitor service · f6bd994
• Make the metrics service to cover the prometheus report · 8d5544d
• Make the heartbeat task in the independent file · b740780
• reorganize log service · 238df43
• Output picked up events size distribution: max + min · 1ae7c76
• Optimize logs and format style · f20731f
• Adjust log level and with more detailed info · ed97597
• Adjust await tree and log · c89c087
• Ignore sensitive info · f3d987a
• Remove opendal crate due to the data loss · 28f7cf3
• Remove sensitive info · 810eb8a
• Remove associated grpc/urpc runtime config · 2d2c98d
• Rename the disk read/write/append metrics names · ce9a5ea
• Disable too much logs about ticket requirement · c9c53d3
• Remove hdfs feature for default features · 95beaab
• No need to use debug format · e97c01b
• Activate default system allocator · 4b321e5
• remove too much logs about creating localfile dir · c4b12db
• Update README · ba5fe36
• Update partial uniffle+netty 300g benchmark result · 99c3825
• Remove unused code · 4584b99
• log the dir creation · 2507e3b
• Update the README · aa36664
• Ignore protobuf generate file · 21dd2f9
• update the benchmark for urpc 300g · 6a4237d
• update the benchmark result for 10g urpc · 1c3e1b7
• ignore the action of build.yaml · 228679f
• log level from info -> debug · 008e1a7
• update the benchmark result · 3fd4505
• setup the constant for initial buffer capacity · bba29d7
• more migration first time · 7a3efc1
• Output the grpc parallelism · af502c1
• feat(localfile)(1/N): Align with multi disk for one partition mechanism · 1ac12dd
• Revert "feat: Apply the unified rejection policy gateway" · 1c3caf7
• Correct disk healthy check min number config key · ac655f9
• feat(disk)(5/N): Activate healthy check · fcec16a
• feat(disk)(4/N): Delete the legacy disk impl · 37c895d
• feat(disk)(3/N): Replace the legacy disk impl · bd7872e
• feat(disk)(2/N): Add the unified disk metrics in delegator abstract layer · 2a7585c
• feat(disk)(1/N): Introduce the pluggable io alternative strategies · c811241
• Add the writing pre-check with crc/lenght (should be removed in the future) · 9c36c74
• Filter out zero buffer when lookup spill buffers · d1250dd
• Isolate localfile + hdfs writing runtime · 015d2d3
• draft(event_bus)(part-5): Add concurrency limit for event handling at the same time · 1bf9900
• draft(event_bus)(part-4): Share the event publish method in hybrid store · fca9d3a
• draft(event_bus)(part-3): Implement and optimize eventbus metrics and logic · 941a7bb
• draft(event_bus)(part-2): Remove the legacy spill queue code · 71400e8
• draft(event_bus)(part-1): Introduce the spillEventHandler · 25761f7
• Complete the stat method for disk to replace get_file_len · b71e13a
• For urpc, bound thread into socket for multiple cores scale performance · 8f29516
• Bound thread into socket for multiple cores scale performance · 52d12d9
• Add more buckets for transport/process time histogram · a1ddd0e
• register the grpc bytes freeze metric when getting memory data · 58cb8de
• Isolate IO tasks to independent runtime · 5e2ee8c
• Log the ticket for debug · f595d88
• optimize memory profile · 16b933a
• update README · 9e90751
• Update benchmark · 7419254
• Update README · 7dd7dba
• Update benchmark result · 824e9e5
• zero copy roadmap · f784031
• add stream incorrect error type · 946c012
• throw error log when parsing frames · 7e2c0be
• fix host public · a0dcdcb
• log accepted connection infos · 9b2aa84
• create the simple config construction · e906121
• correct log · ec1c3fc
• correct to urpc · b97fc80
• create the unified rpc service for urpc + grpc · 5a3cb38
• reconstruct project structure · c68f73d
• implement customize protocol · 9d232ff
• try to change the run-on system · 1f9476c
• try to build artifact · c4cb733
• Add ROADMAP · 8e62104
• Update readme · 361f148
• update the readme · 8096a41
• Use rwlock to replace mutex in localfile · 58936e7
• ensure the only one partition writer spill to file · 7b5bf0c
• fix some tests compile failure · a900902
• Update the benchmark report · 1af6b4d
• Merge multi allocated->used operations into one · 8c3c300
• disable fastrace · 0f26c66
• Use mutex to replace rwlock for budget · 15ba095
• correct spill usage ratio logic · 46cb583
• avoid outputting too much unnecessary logs · 89db0db
• take with blocks len to avoid used metric incorrectness · 3d01dc4
• avoid spin lock · 4754dab
• avoid fastrace in critial path · 01fcdbb
• use parking lot to avoid spin lock · 5e319d2
• Build the binary by action · f8e2378
• Push events into jaeger · a34c6de
• Init Fastrace · 51a35ae
• introduce tracing · 491ad31
• adjust format · bcbfe01
• Remove async watermark flush · 78a3458
• Add unit tests for memory budget · 499bb4e
• rename some methods · be68927
• Restructure the memory buffer · b649d37
• avoid string recursion format inner · dfa29de
• avoid clone uid · 6bd4c2d
• try to lock-free for budget · 045dc28
• try to use vec to replace linkedlist · 407fa1f
• use env to specify shard number · 7e58cca
• shuffle sending blocks to avoid conflict · af8e4a4
• try to use the std rwlock · ba6d910
• recover · 445514f
• try to use contiguous bytes · 0fe2eea
• try zero_copy features · b5a1772
• remove all logic code to test the real tonic throughput · e231757
• try to use vec.extend to replace one-by-one insert · 34314c8
• fix · cded155
• try to log pick time · 0c9e6e0
• draft-2 · 0a229ea
• draft · 95dfa55
• try again · c49f916
• try optimize vec.clear · 8cdd9bb
• record clear time · 639f44f
• record insert time · 6e7c7fa
• add migration time · 11c112e
• Use the sync io to replace async io · bc59f45
• Remove console-subscribe to avoid potential memory leak · db5adc7
• Add CI tests pipeline · 1261905
• Initial commit that is copied from the Apache Uniffle’s rust code base · 50caf17