Benchmark

The Riffle repository reports a TeraSort comparison using a 273 GB compressed dataset. The numbers below retain the original environment and configuration context.

Environment

Type	Description
Software	Uniffle 0.8.0, Hadoop 3.2.2, Spark 3.1.2
Machine	96 cores, 512 GB memory, 4 × 1 TB SATA SSD, 8 GB/s network
Hadoop YARN cluster	1 ResourceManager + 40 NodeManagers
Uniffle cluster	1 coordinator + 1 shuffle server

Spark configuration

spark.executor.instances=400
spark.executor.cores=1
spark.executor.memory=2g
spark.shuffle.manager=org.apache.spark.shuffle.RssShuffleManager
spark.rss.storage.type=MEMORY_LOCALFILE

TeraSort result

Each bar shows total runtime, split into shuffle write and shuffle read time. Lower is better.

Shuffle write Shuffle read Total Runtime vs Spark ESS

Vanilla Spark ESS

1.3 2.9

4.2 min

—

Riffle gRPC 10 GB

1.9 2.1

4.0 min

−4.8%

Riffle gRPC 300 GB

1.4 2.1

3.5 min

−16.7%

Riffle uRPC 10 GB

1.6 2.2

3.8 min

−9.5%

Riffle uRPC 300 GBFastest

1.2 2.0

3.2 min

−23.8%

Uniffle gRPC 10 GB

1.8 2.2

4.0 min

−4.8%

Uniffle gRPC 300 GB

2.7 5.9

8.6 min

+104.8%

Uniffle Netty, default allocator 10 GB

2.7 2.4

5.1 min

+21.4%

Uniffle Netty, jemalloc 10 GB

2.0 2.5

4.5 min

+7.1%

Uniffle Netty, default allocator 300 GB

1.5 2.5

4.0 min

−4.8%

Uniffle Netty, jemalloc 300 GB

1.9 4.7

6.6 min

+57.1%

The Riffle uRPC protocol is named NETTY on the Java side.

These figures are project-reported measurements, not a guarantee for another cluster. Compare with the same Spark version, data shape, network, disks, storage capacity, and concurrency before drawing conclusions.

See the source README for the original report.